W4 Null-Hypothesis Significance Testing

Question 1

What can we do to reduce Type I errors?

Answer 1

Lower α threshold (see reading list) The downside is a higher β. If the null hypothesis is true, then the probability of making a Type I error is equal to the significance level of the test. To decrease the probability of a Type I error, decrease the significance level.

Question 2

What can we do to reduce Type II errors?

Answer 2

β is directly related to 1 - β (obviously).

Thus, if we increase 1 - β, β decreases 1 - β is power

Question 3

What happens if we perform more than one test?

Answer 3

Probability of Type I error increases

• 1-(1-.05)^n, with n = number of comparisons

Multiple methods are available to correct for α inflation

• e.g. Bonferroni (α / n, with n = number of comparisons)
• but also others: Tukey, Scheffé, FDR, etc.