Power & Blocking Designs

Question 1

What is a a Type 1 error?

Answer 1

Incorrectly rejecting the null hypothesis (false positive)
Finding a significant difference in the sample that doesn’t exist in the population

Question 2

What is a Type 2 error?

Answer 2

Incorrectly retaining the null hypothesis (false negative)
Finding no significant difference in the sample when one exists in the population

Question 3

What is power?

Answer 3

Correctly rejecting the null hypothesis
The degree to which we can detect treatment effects when they exist in the population

Question 4

What is the probability of making a Type 1 error?

Answer 4

5%
α = .05

Question 5

What is the probability of correctly retaining the null hypothesis?

Answer 5

95%
1 - α

Question 6

How is power calculated?

Answer 6

1 - β

Question 7

What is the probability of making a Type 2 error?

Answer 7

β

Question 8

What are 4 ways to increase power?

Answer 8

1. Increase sample size
2. Increase alpha (but increases Type 1 error rate)
3. Find a larger effect
4. Reduce error term

Question 9

What are 4 ways to increase power?

Answer 9

1. Increase sample size
2. Increase alpha level (but increases Type 1 error)
3. Focus on larger effects
4. Decrease error variance

Question 10

Why would you care about power before running a study?

Answer 10

To make sure there is enough power to detect predicted effects

Question 11

Why would you care about power after running a study?

Answer 11

To increase power to detect a difference in a population that is there but was not observed in the sample

Question 12

What are estimates needed to calculate power a priori?

Answer 12

1. Estimate of effect size
2. Estimate of error (MSerror)
   Typically using previous research

Question 13

What values are needed to calculate power post-hoc?

Answer 13

1. Effect size
2. Error (MSerror)
3. N in the study
   From the dataset

Question 14

What is effect size?

Answer 14

It measures the overlap of distributions in terms of standard deviations and is closely related to power

Question 15

What are ways to decrease MSerror?

Answer 15

1. Improve operationalisation of variables (increase validity)
2. Improve measurement of variables (increase reliability)
3. Improve design of study by accounting for variance from other sources (blocking)
4. Improve methods of analysis by controlling for variance from other sources (ANCOVA)

Question 16

What do blocking designs do?

Answer 16

They introduce a control variable to account for additional variance in the DV. It increases power by reducing error variance

Question 17

How do you set up a blocking design?

Answer 17

1. Divide participants into groups to make a continuous variable ‘categorical’ (low, med, high)
2. Randomly assign participants within each block to levels of the IV

Question 18

What is stratified random assignment?

Answer 18

Randomly assigning participants within each level of the blocking factor to different levels of the IV

Question 19

What is a sign that the blocking variable is a good control variable?

Answer 19

If there is a main effect of the blocking variable

Question 20

What is a sign that the blocking variable is a confound?

Answer 20

Is there is an interaction between the blocking variable and the IV

Question 21

What are some advantages of blocking?

Answer 21

1. Greater power
2. Can check interactions for potential confounds
3. Equates treatment groups better than a completely randomised design

Question 22

What are some disadvantages of blocking?

Answer 22

1. Practical costs
2. Loss of power if poorly correlated with the DV
3. Grouping can cause loss of information