Power Analysis

Question 1

what do we do when the F-ratios that exceed F-critical?

Answer 1

• we reject the null hypothesis.
• Independent variable is related to the dependent variable.
• Result: Statistically significant effect.

Question 2

what do we do when the F-ratio does not exceed F-critical?

Answer 2

• fail to reject the null hypothesis.
• H0 : Means are equal; no evidence that IV is related to DV
• Result: No statistically significant effect

Question 3

what is statistical power?

Answer 3

the probability of detecting a true (population) effect given a particular sample.

Question 4

how is power defined in an equation?

Answer 4

1 – 𝛽

Question 5

what is where 𝜷 ?

Answer 5

where 𝜷 is the probability of making a type-II error

Question 6

what three factors determine the power of a study?

Answer 6

• Alpha level
• Sample size (𝑛)
• Effect size

Question 7

what is a type-II error?

Answer 7

the probability of failing to reject the null hypothesis when it is false

Question 8

how does a difference in population means effect power?

Answer 8

• increased difference in population means = more power, bigger effect in size

Question 8

how does difference in population variance effect power?

Answer 8

Decreased population variance (same mean difference) → more power, bigger effect size

Question 9

what does effect size indicate?

Answer 9

The practical significance (or magnitude) of a research outcome.

Question 10

what are the measures of association?

Answer 10

• Eta-squared (n2)
• R-squared (R2)

Question 11

what are the measures of difference?

Answer 11

Cohen’s 𝒅

Question 12

what is eta squared?

Answer 12

the proportion of variance in DV explained by a single IV

Question 13

what is a partial eta-squared?

Answer 13

similar, but estimates a specific effect size if there are more than one IV

Question 14

what is R sqaured?

Answer 14

• The proportion of variance explained by the model.
• 𝟏 minus the proportion of variance unexplained

Question 15

what is cohen’s d?

Answer 15

When there are only two groups, 𝒅 is a ratio of the difference between the two groups with their error variance

Question 16

what are the two ways to decide what effect size is being aimed for?

Answer 16

• based on previous research: Meta-analysis: Review previous literature and calculate previously observed effect size (from the same or similar studies)
• Based on theoretical importance: Deciding whether a small, medium, or large effect is required

Question 17

why is power analysis important for the estimation of the sample size?

Answer 17

• If a sample size is too small, a true effect may be missed
• But larger sample sizes are more expensive

Question 17

How do we estimate sample size?

Answer 17

• Once we have values for effect size and alpha, we can use a software tool called G*Power
• This estimate the minimal sample size we would need to obtain a specific power level.
• If we desire 80% power (i.e., 80% of the time, our sample will find a true effect if it exists), we input it into G*Power with the effect size and alpha to get our sample size estimate

Question 18

how do we calculate the sample size based on previous research?

Answer 18

Example based Foa et al.
- Participants were 48 trauma victims who were randomly assigned to one of four treatment groups:
1. Stress Inoculation Therapy (SIT) in which subjects were taught a variety of coping skills;
2. Prolonged Exposure (PE) in which subjects went over the traumatic event in their mind repeatedly for seven sessions;
3. Supportive Counseling (SC) which was a standard therapy control group
4. Waiting List (WL) control
- Dependent variable was post-traumatic stress (PTSD) severity