Lecture 13; One-Sample Testing:

Question 1

What is the core problem in statistics about never knowing the truth?

Answer 1

We do not know whether a hypothesis is actually true or false (e.g., Jinder’s coin is or not); we only observe sample data and make a decision based on those data.

Question 2

What does statistics provide a framework for?

Answer 2

Statistics therefore provides a framework for reasoning and making decisions under uncertainty, not a way of directly discovering the truth.

Question 3

In statistics, two things exist simultaneously, what are they?

Answer 3

1) The real truth about the hypothesis (which we cannot observe).
2) Our decision based on the data (reject or not reject H₀). Because we do not know the truth, our decision can be correct or incorrect. By rejecting null hypothesis, you aren’t providing evidence against the null hypothesis

Question 4

What is a type 1 error?

Answer 4

Type I error (false positive): Null is correct, but reject the null

Question 5

What is a type II error?

Answer 5

Type II error (false negative): Null is wrong, but the null is not rejected

Question 6

True or false: statistical testing guarantees correct conclusions

Answer 6

FALSE: statistical testing cannot guarantee correct conclusions

Question 7

What does it mean when the p-value is large?

Answer 7

When the P-value is big = fits with the null hypothesis

Question 8

What does it mean when the p-value is small?

Answer 8

When P-value is small = demonstrates significance meaning reject the null hypothesis

Question 9

Is statistical testing qualitative or quantitative?

Answer 9

Statistical hypothesis testing is a quantitative inference framework.

Question 10

What does statistical hypothesis testing evaluate?

Answer 10

It evaluates how compatible the data are with an assumed model = the null hypothesis (HO)

–> Core idea: we evaluate how surprising the observed data would be if the null hypothesis (H₀) were true.

Question 11

How does one decide on when to reject the null hypothesis?

Answer 11

The significance level, denoted by 𝛼 (alpha), is the threshold we set before analyzing the data to decide how much incompatibility with the null model we are willing to tolerate before rejecting it.

Question 12

What does estimation ask and what does hypothesis testing ask?

Answer 12

1) Estimation asks - How large is the effect?
2) Hypothesis testing asks - Is there any effect at all?

Question 13

Why do people prefer statistical hypothesis testing over estimation?

Answer 13

• It gives us binary answers
• Society understands much better yes or no decisions = qualitative

Question 14

What does statistical hypothesis testing focus on?

Answer 14

Statistical hypothesis testing does not focus on the exact proportion value, but on whether there is evidence that the proportion differs from a specified value

Question 15

What is the P-value?

Answer 15

The p-value is the probability, calculated under the assumed null hypothesis (H₀), of observing a value of the test statistic (θ) as extreme as, or more extreme than, the one actually observed

Question 16

Why we are the ones that set type I error (alpha) but not type II error (beta)?

Answer 16

Because α is defined from the probability (sampling) distribution assuming H0 is true, it represents the probability of committing a Type I error (a false positive); that is, rejecting H0 when H0 is actually true.

Question 17

What are the highlighted tails of the t-distribution related to statistical hypothesis testing?

Answer 17

*The highlighted areas under the curve are values that are rare that aren’t likely to occur and thus demonstrate that there might be an effect, so you reject the null hypothesis

Question 18

In statistical hypothesis testing, we construct the sampling distribution under the assumption that…

Answer 18

…the null hypothesis (H0) is true.
–> This means that all values in that distribution, including the observed sample value, are outcomes that could occur if H0 were true.

Question 19

Where must a value lie on the distribution such that we reject the null hypothesis?

Answer 19

If the observed sample value lies in a region of the distribution that is sufficiently unlikely under H0, we conclude that the result is improbable under the null model and reject H0.

Question 20

What is the protection against incorrectly rejecting a true null hypothesis (a Type I error) determined by?

Answer 20

The protection against incorrectly rejecting a true null hypothesis (a Type I error) is determined by the chosen alpha level (the significance level).

Question 21

How can you reduce the probability of failing to reject a false null hypothesis (Type II error)?

Answer 21

Typically requires increasing statistical power, often by increasing the sample size.

Question 22

When does a Type I error occur?

Answer 22

A Type I error occurs when a true null hypothesis is incorrectly rejected (i.e., rejecting the null hypothesis when it should not be rejected). Its probability is the significance level (α), which is determined by us and remains unaffected by the sample size (n)

Question 23

When does a Type II error occur?

Answer 23

Type II error is failing to reject a false null hypothesis (i.e., do not reject the null hypothesis when you should not have). Its probability is β and is more complex to estimate (advanced stats). This probability decreases as sample size increases.

Question 24

What is the power of a test (1 - β)?

Answer 24

The power of a test (1 - β) is the probability of correctly rejecting the null hypothesis when it is truly false. This probability increases as the sample size grows.

Question 25

What is the t-value?

Answer 25

t-values = the number of standard errors that the sample estimate is away from the theoretical parameter assumed under H0

Question 26

What occurs to the t-distribution if the population isn't normally distributed?

Answer 26

If the original population is not normally distributed, the standardized sampling distribution of the sample mean may also deviate from the expected t-distribution, and the test may not perform as assumed (as discussed earlier for confidence intervals).

Question 27

What is the t-distribution used for?

Answer 27

Unlike discrete variables (e.g., handedness in toads), the t-distribution is used for continuous variables (e.g., temperature) and is described by a probability density function (pdf). Because the probability of any exact value is zero, probabilities are calculated as the area under the curve between two values.

Question 28

What is the probability of obtaining a sampling mean as extreme or more extreme than 98.524oF given that the theoretical population mean (assumed under H0) is 98.6oF? --> The sample mean is -0.56 standard deviations away from the mean of the theoretical population (assumed under H0) --> Should we reject or not reject the H0?

Answer 28

The probability of observing a t-value at least as extreme as ±0.56 is 0.58. The P-value is greater than the threshold so we do not reject the null hypothesis.

Question 29

What is The Procedure for a One-Sample Mean Test in 5 steps?

Answer 29

1) Specify the mean under H0. 2) Take a sample from the population of interest. 3) Standardize the sample mean using the t-statistic. 4) Calculate the probability of observing the t-value or a more extreme value in the t-distribution. 5) Compare the probability with alpha and make a decision about H0.

Question 30

Imagine the most compatible result possible: If the sample mean were exactly equal to the null hypothesis (98.6°F): What is the p-value and what does it mean?

Answer 30

P-value would = 1.00 --> What does that mean? The data are completely compatible with H0. But even in this case: We still cannot conclude that H0 is true. --> We can only say: The data provide no evidence against it.

Question 31

True or false: Statistical tests prove hypotheses

Answer 31

FALSE: Statistical tests do not prove hypotheses, they only evaluate whether the data contradict them.

Question 32

What could be the 2 possible results under this question: What is the probability of obtaining a sampling mean as extreme or more extreme than 98.524oF given that the theoretical population mean (assumed under H0) is 98.6oF?

Answer 32

1) If consistent (large P-value), then we can state that we have no evidence to state that the human temperature is different from 98.6oF. 2) If inconsistent (small P-value), then we would have stated that we have evidence that the Normal human body temperature is not 98.6oF.