Scheffé’s
post-hoc test (more conservative, less power than Tukey’s)
Tukey’s HSD
post-hoc test (more power than Scheffé’s)
Bonferroni
post hoc, adjusting Type I error rate and critical value (dividing by number of tests)
Sidak
post hoc, less conservative than Bonferroni correction, also adjusts Type I error rate
Dunnet
post-hoc test, comparing 1 group to the other k-1 groups
Holm
post hoc, sequential mean comparisons using Bonferroni correction
Fisher-Hayter
post hoc, starts with largest mean difference and keep going until H0 is retained
using Qcrit with df = k-1 (less conservative than Tukey)
Newman-Keuls
post hoc, starts with largest mean difference and keep going down until H0 is retained
minimum absolute difference is re-calculated for every comparison
Duncan
post hoc, starts with largest mean difference and keep going down until H0 is retained
minimum absolute difference is re-calculated for every comparison (same as Newman)
uses Sidak’s Fcrit
post hoc tests
- Scheffé
- Tukey’s HSD
- Bonferroni correction
- Sidak’s correction
- Dunnet
- Holm
- Fisher-Hayter
- Newman-Keuls
- Duncan
tests for normality
- test for skewness
- Kolmogorov-Smirnov (quantiles)
- Shapiro-Wilk (quantiles)
- Q-Q plots
- histograms
tests for homoscedasticity
- Hartley’s F-max
- Levene’s test (ANOVA on deviation scores from group means)
- Brown-Forsythe (ANOVA on deviations from the group medians)
ways to correct a violation of homoscedasticity
- run anova on sqrt(outcomes) - weak
- run ANOVA on log(outcomes) - mild
- run ANOVA on 1/outcomes - strong
- Box-Cox transformation
- non-parametric tests (Kruskal-Wallis test uses medians)
Effects of a mixed design
- between-subjects (main effect of A - subject to assumption of homoscedasticity, normality, independence)
- subject variation within levels of A (residual for between-subjects)
- within-subjects factor (main effect of B - subject to assumption of sphericity and normality)
- interaction effect between A and B
- interaction between within-subject factor B and subjects nested within levels of between-subjects factor A (residual for main effect of B and interaction)
effect of violating normality
decrease in Type I error rate than nominal (less power)
effect of violating homoscedasticity
increase in nominal Type I error rate
define residual variance in between-subjects ANOVA
variance within groups (random fluctuations in subject scores)
orthogonal comparisons
independent portions of variance due to group membership (limited number of comparisons k-1, once all are computed = model variation SSm)
non-orthogonal comparisons
could deal with overlapping pieces of the model variation, so could amount to more than SSm
follow-up to a two-way ANOVA with no significant interaction
comparison of marginal means
follow-up to two-way ANOVA with a significant interaction that is dominated by main effects
comparison of marginal means
follow-up to two-way ANOVA with a significant interaction that dominates main effects
simple main effects
if those are significant, follow them up with simple comparisons if you have more than two levels (i.e. directionality not obvious based on cell means)
relationship between Type I error and power
increasing nominal Type I error rate (less conservative alpha) = increasing type I error rates and increasing power
- decreasing nominal Type I error rate (more conservative alpha) = decreasing type I error rates and decreasing power
assumptions of one-way repeated-measures
- normality of DV in the population
- homogeneity of variances
- homogeneity of covariances
assumptions 2&3 are compound symmetry - assumption of sphericity
