Chapter 8

Question 1

Hypothesis Testing

Answer 1

A statistical method that uses sample data to evaluate a hypothesis about a population

Question 2

Hypothesis

Answer 2

A specific testable prediction about the association between two variables

Question 3

What is the general goal of hypothesis testing?

Answer 3

To rule out chance (sampling error) as a plausible explanation for the results from a research study

BUt keep in mind that it doesn’t actually do that

Question 4

Hypothesis testing uses probability to support one of two possible explanations for our findings

Answer 4

1. The observed findings are due to random change (there dows not appear to be a real effect)… called the null hypothesis
2. The observed findings cannot be explained by sampling error (there does appear to be a real effect)… called the alternative hypothesis

Question 5

Hypothesis Test: Step 1

Answer 5

State your hypothesis about the population
* The null hypothesis, H0, predicts that the indepenent variable had no effect on the dependent variable
* The alternative hypothesis, H1, predicts that the independent variable did have an effect on the dependent variable

Question 6

The Hypothesis Test: Step 2

Answer 6

Define the probability at which we think results indicate that there must be a true effect
* The a level establishes a criterion, or “cut-off,” for deciding if the null hypothesis is correct (typically a=.05)
* Critical region consists of outcomes very unlikely to occur if the null hypothesis is true (Defined by associations that are very unlikely to obtain (typically less than 5% chance) if no effect exists)

Question 7

a (alpha) level

Answer 7

ha

Question 8

Critical region

Answer 8

consists of outcomes very unlikely to occur if the null hypothesis is true
-Defined by associations that are very unlikely to obtain (typically less than 5% chance) if no effect exists

Question 9

They Hypothesis Test: Step 3

Answer 9

• Obtain and test a sample from the population
• Compute the relevant test statistic

Question 10

Test Statistic

Answer 10

forms a ratio comparing the difference between the population mean and sample mean versus the amount of difference we would expect without any treatment effect (standard error of the M)

ex. Z-score

Question 11

The Hypothesis Test: Step 4

Answer 11

Compare data with the hypothesis predictions
-If the test statistic results are in the critical region, we conclude the difference is significant (an effect exists) and we reject the null hypothesis
-If the test statistic is not in the critical region, conclude that the difference is not significant (any difference is just due to chance), we fail to reject the null hypothesis

Question 12

Errors in Hypothesis Tests

Answer 12

• Hypothesis tests are not perfect. Evem when a test statistic falls in the critical region, we aren’t certain an effect exists
• Remember: we always expect some discrepancy between a M and u, there is a risk that misleading data will cause the hypothesis test to reach a wrong conclusion, two types of errors are possible

Question 13

Type 1 Errors

Answer 13

Occur when the sample data indicate an effect when no effect actually exists; rejecting the null hypothesis when the null is true
* Caused by unusual, unrepresentative samples, falling in the critical region without any true effect
* Hypothesis tests are structured to make Type 1 errors unlikely

Question 14

Type 2 Errors

Answer 14

Occur when the hypothesis test does not indicate an effect but in reality an effect does exist
* We fail to reject the null hypothesis even though it was actually false
* More likely with a small treatment effect or poor study design (sample size too small)

Question 15

Directional test (or one-tailed test)

Answer 15

• Includes a directional prediction in the statement of the hypotheses and in the location of the critical region
• Ex: The original population has u=75 points on a stats test and studying is predicted to increase the scores
  -HO: Test scores are not increased
  -H1: Test scores are increases
  -Entire critical region would be located in the positive tail bc large values for M would demonstrate that there is an increase and we would reject the null hypothesis (z=1.645+)

Question 16

P-values (probability values)

Answer 16

• When performing hypothesis tests we will check statistical significance by seeing if our test scores (ex. z-score) indicate a p-value of less than out a level
• -Ex. we set aplha at a=.05 and check to see if p<.05

Question 17

P-value definition

Answer 17

• The probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis

Do not tell us the probability that we’re making a Type 1 Error

Question 18

Interpreting p-value correctly
Ex. Imagine we’re testing a vaccine and out hypothesis test yields p=.05

Answer 18

• Incorrect: If you reject the null hypothesis, there’s a 5% chance that you’re making a mistake (is actually probably around 20-50% depending on context)
• Correct: assuming the vaccine had no effect, you’d obtain the observed difference or more in 5% of stdies due to randome sampling error

Question 19

What does a hypothesis test evaluate?

Answer 19

The statistical significance of the results from a research study

Question 20

What influences hypothesis test?

Answer 20

The size of the treatment effect and the size of the sample… even a very small effect can be statistically significant if observed in a very large sample (would have a very small standard error)

Question 21

Effect Size

Answer 21

-Measure of the absolute magnitude of an effect, independent of sample size
-Hypothesis tests should be accompanied by effect size

Question 22

Cohen’s d

Answer 22

• Is a standardized effect size
• Like a z-test, Cohen’s d measures mean difference in terms of the standard deviation

M-u/o

Question 23

Cohen’s d effect sizes

Answer 23

• d=.2-.49… small effect
• d=.5-.79… med effect
• d=.8+… large effect

Question 24

Power of a hypothesis test

Answer 24

• The probability that the test will reject the null hypothesis when there is actually an effect
• Likelihood we can find what were looking for depends on:
  -Effect size (larger effects are easier to find)
  -Sample size (larger samples make it easier to find effects)
  -Alpha level (larger alpha level makes it easier to find effects)
  -Non-directional vs directional hypothesis (directional tests make it easier to find effects)

Question 25

If other factors are held constant, then how does sample size affect the likelihood of rejecting the null hypothesis and the value for Cohen’s d?

Answer 25

A larger sample increases the likelihood of rejecting the null hypothesis but has no effect on the value of Cohen’s d

Question 26

Under what circumstances is a very small treatment effect most likely to be statistically significant?

Answer 26

With a large sample and a small standard deviation

Question 27

test statistic

Answer 27

indicates the sample data are converted into a single, spedific statistic that is used to test hypotheses

Question 28

Statistical significance tells us what?

Answer 28

That the results are unlikely to have occurred if there is no treatment effect