When is unequal N too extreme?
Ratio of largest to smallest cell size > 3:1
In what ways do highly unequal N make analyses more unstable?
They increase both type 1 and 2 error.
def. power
Probability of correctly rejecting a false null hypothesis.
What are four interdependent factors affecting power?
- Significance level
- Sample size
- Mean difference
- Error variance
If you want to study small effects it will be highly likely you need ___ ?
Large N / sample size
What is the best way to determine power?
A priori
When can post hoc power analyses be useful? (2)
- In light of non-significance, to show a significant result would have required much larger N for the observed effect size.
- In light of non-significance, to show there was sufficient power to suggest a true case for the null.
A priori power estimates ask the question …
What N to achieve a given (.8) power?
Post hoc power estimates ask the question …
What power did I have given my N and effect size
What are four things we can improve to reduce error variance in a study?
- Operationalisation of variables (validity)
- Measurement of variables (internal reliability)
- Design of study - account for variance from other sources (blocking)
- Methods of analysis - control for variance from other sources (ANCOVA)
What are two applications of blocking?
- Reducing error variance, where the focal IV is underpowered.
- Detecting confounds, in the presence of a block-factor x IV interaction.
Would you expect a main effect of a blocking factor?
yes. This is a sign of good control variable, I.e. related to DV, but not IV.
What does it mean if a IV treatment effect does not generalise across levels of a blocking factor?
There is an interaction between IV and blocking factor, therefore a confound.
What is one advantage and disadvantage of a block x IV interaction?
Pro: the interaction soaks up more systematic variance, making the focal main effect more significant.
Con: this is outweighed by the focal IV effects DEPENDING on the moderator/confound (I.e. blocking factor)
When is there a loss of power due to including a blocking variable?
When there is a low correlation with block factor and DV, r < .2. (because of fewer error degrees of freedom)
How does blocking inadvertently lose power if the blocked factor is not related to the DV?
It reduces the df in MSerror due to calculating means for another set of cells. MSerror will then be higher, reducing power.
What is the difference between a control variable and a confound variable?
The latter is just unwanted/unpredicted.
