Sample Size And Study Power

Question 1

What is the purpose of a power calculation?

Answer 1

It helps you choose a sample size such that if the new drug truly is substantially better, we would be fairly certain of getting a significant result

Question 2

What are the two different approaches to sample size calculations?

Answer 2

1. Precision approach

2. Power approach

Question 3

When is a precision approach used for study size calculations?

Answer 3

If the study aim is to obtain a prevalence estimate (or other estimate) and 95% CI

You need a rough guess of the prevalence and an idea of how precise or narrow you want the confidence interval to be

Question 4

When would you need more people in your sample for a precision approach?

Answer 4

• If the prevalence is closer to 0.5 (50%)
• if you want a narrower confidence interval

e.g.
Prevalence of 8% and want to estimate within 2% of truth (with 95% confidence) = need 706 people
Prevalence of 10% and want to estimate within 2% = need 864 ppl
Prevalence of 10% and want to estimate within 1% = need 3445 ppl

Question 5

When is a power approach used for sample size calculations? What do you need to know?

Answer 5

If the study aim is to carry out a statistical test to compare two groups

You need to know:

1. a rough guess of the prevalence (% with outcome) in baseline group
2. Minimum difference/effect you want to be able to detect
3. Set the probability of Type I error (usually 5%)
4. Set the probability of Type II error (1 minus this = study power). Power is the prob of detecting an effect as significant if it really exists (usually 80-90% used)

Question 6

When would you need more people in your sample for a power approach?

Answer 6

When you want to detect a smaller difference between the groups, want a smaller significance level, want greater power or if the population prevalence is closer to 0.5

E.g.
Prevalence 4% in women. Want to detect difference of 2% higher in men with 90% power and at 5% sig level = 2600 in each group

Same as above but with 80% power = 1960 in each group

Question 7

What are some general points for study size calculations?

Answer 7

• Only rough estimates
• Try different scenarios
• Increase sample size to allow for non-response/drop out
• matching (case control study) can increase power
• can do it for unequal sized groups

Question 8

Does a non-significant result mean no true effect?

Answer 8

Not necessarily - may be that the sample size is too small

Research papers should always provide details of the power/sample size calculations in the methods section but this isn’t always the case

Question 9

What are the downsides of recruiting too many or too few participants?

Answer 9

Too many = waste of resources

Too few = may fail to detect important effect and estimates of the effect may be too imprecise (wide CIs)