Testing ANOVA assumptions

Question 1

A Type I error has been defined as the

Answer 1

probability of rejecting the null hypothesis when in fact the null hypothesis is true
•This applies to every statistical test that we perform on a set of data.

Question 2

If we perform several statistical tests on a set of data we can effectively

Answer 2

increase the chance of making a Type I error.

Question 3

If we perform two statistical tests on the same set of data then we have a…

Answer 3

range of opportunities of making a Type I error
•Type I error on the first test only
•Type I error on the second test only
•Type I error on both the first and the second test

Question 4

What are per comparison errors?

Answer 4

Type I errors involving single tests

Question 5

What are familywise errors?

Answer 5

A whole set of type 1 errors

E.g. •Type I error on the first test only
•Type I error on the second test only
•Type I error on both the first and the second test

Question 6

The relationship between pre comparison and familywise error rates is very simple:

Answer 6

Afw = c ( Apc)

Where c is the number of comparisons

• So if we have made three comparisons, we can expect 3(0.05) = 0.15 errors. If we make twenty comparisons, we will on average make one error [200.05=1.0].
• Of course, if we make twenty comparisons, it is possible that we may be making 0, 1, 2 or in rare cases even more errors.

Question 7

With planned comparisons :

Answer 7

Ignore the theoretical increase in familywise type I error rates and reject the null hypothesis at the usual per comparison level.

Question 8

With post hoc or unplanned comparisons between the means:

Answer 8

we cannot afford to ignore the increase in familywise error rate

Question 9

A variety of different post hoc tests are commonly used - for example:

Answer 9

• Scheffé
• Tukey HSD
• t-tests
• These tests vary in their ability to protect against Type I errors.
• Increasing Type I protection reduces Type II protection.

Question 10

The Scheffe Test

Answer 10

• The Scheffé is calculated in exactly the same way as a planned comparison
• Scheffé differs in terms of the FCritical that is adopted.
• For the one-way between groups analysis of variance the critical F associated with an FScheffé is given by:
• where a is the number of treatment levels and F(dfA, dfS/A) is the critical value of F for the overall, omnibus analysis of variance.
• For our example
• Omnibus ANOVA critical value F(2,12)= 3.885. There were three treatment levels so (3-1)*3.885= 7.77.
• Fobserved = 14.29

Question 11

Tukey HSD

Answer 11

• The Tukey (Honestly Significant Difference) test establishes a value for the smallest possible significant difference between two means.
• Any mean difference greater than the critical difference is significant
• The critical difference is given by:
• where q(a,df,a) is found in tables of the studentized range.
• This particular formula only works for between groups analysis of variance with equal cell sizes
• A variety of different formulae are used for different designs

Question 12

What is a Bonferroni correction?

Answer 12

When comparing two means, a modified form of the t-test is available.

• For multiple comparisons the critical value of t is found using
• p=0.05/c
• where c is the number of comparisons.

Question 13

When comparing two means, a:

Answer 13

modified form of the t-test is available

Question 14

For multiple comparisons the critical value of t is found using:

Answer 14

• p=0.05/c

* where c is the number of comparisons

Question 15

Post Hoc Tests

Answer 15

• Post-hoc tests are conservative – they reduce the chance of type I errors by greatly increasing type II errors.
• Only very robust effects will be significant.
• Null results using these tests are not easy to interpret.
• Many different post hoc tests exists and have different merits and problems
• Many post hoc tests are available on computer based statistical packages (e.g. SPSS or Experstat)

Question 16

Post Hoc tests are conservative, this means…

Answer 16

• they reduce the chance of type I errors by greatly increasing type II errors.
• Only very robust effects will be significant.

Question 17

Null results using post Hoc tests are not…

Answer 17

Easy to interpret
•Many different post hoc tests exists and have different merits and problems
•Many post hoc tests are available on computer based statistical packages (e.g. SPSS or Experstat)

Question 18

Many post hoc tests are available on

Answer 18

computer based statistical packages (e.g. SPSS or Experstat)

Question 19

The assumptions of the F ratio

Answer 19

• Independence
• The numerator and denominator of the F-ratio are independent
• Random Sampling
• Observations are random samples from the populations
• Homogeneity of Variance
• The different treatment populations have the same variance.
• Normality
• Observations are drawn from normally distributed populations

Question 20

The assumptions of the F ratio:

Answer 20

Independence
Random sampling
Homogeneity of variance
Normality

Question 21

The assumptions of the F ratio

Independence

Answer 21

The numerator and denominator of the F-ratio are independent

Question 22

The assumptions of the F ratio

Random Sampling

Answer 22

Observations are random samples from the populations

Question 23

The assumptions of the F ratio

Homogeneity of Variance

Answer 23

The different treatment populations have the same variance

Question 24

The assumptions of the F ratio

Normality

Answer 24

•Observations are drawn from normally distributed populations

Question 25

Testing assumptions of ANOVA | Each of these assumption should be met before...

Answer 25

progressing onto the analysis.

Question 26

There are two assumptions of ANOVA that we have to assume have been met by the experimenter...

Answer 26

* Independence and Random Sampling | * If an experiment has been designed appropriately both of these assumptions will be true.

Question 27

Both the homogeneity of variance and the normality assumptions of ANOVA...

Answer 27

need not necessarily be true

Question 28

Testing Homogeneity of variance | When looking a between groups designs use

Answer 28

* Hartley's F-max * Bartlett * Cochran's C * All these tests are sensitive to departures from normality * All of these tests are available in SPSS (as are a number of other tests)

Question 29

Testing homogeneity of variance | When looking at within or mixed designs use

Answer 29

Box's M

Question 30

For hand calculations, there is a quick and dirty measure of homogeneity of variance:

Answer 30

Largest variance / smallest variance Smaller than 4? this is a heuristic. When you have the option, use one of the specific tests (e.g. Bartlett).

Question 31

•The three most commonly used tests for normality are:

Answer 31

* Skew * Lilliefors * Shapiro-Wilks * These tests compare the distribution of the data to a theoretically derived normal distribution. * All these tests are very sensitive to departures from normality when there are large samples.

Question 32

The Lilliefors and Shapiro-Wilks are difficult to

Answer 32

calculate by hand, but both are available on SPSS.

Question 33

Testing normality by examining skew

Answer 33

* Since we assume * that the distributions of the population from which the samples are taken are normal * and the skew of a normal distribution is equal to zero * Then * One test of normality is to see if the skew is significantly different to zero * In other words, test the value of skew to see if it deviates significantly from a normal distribution.

Question 34

The simplest test we can use is ____ to test for skew

Answer 34

Z-score

Question 35

Transforming data reduces the probability of making a

Answer 35

type II error

Question 36

type II error occurs when we fail to

Answer 36

reject the null hypothesis when it is false

Question 37

If an assumption is broken, ANOVA fails gracefully:

Answer 37

we will miss real effects (type II) but we will not increase our rate of making claiming effects that do not exist (type I)

Question 38

Data should be transformed when either:

Answer 38

the data is not homogenous or not normal

Question 39

Solving the homogeneity problem often solves the:

Answer 39

normality problem and vice versa

Question 40

What happens when transforming the data is impossible?

Answer 40

•In general we proceed with the analysis but advise caution to the reader when reporting the results •This is particularly important if the observed F value has an associated probability, p, such that 0.1>p>0.01 •In these circumstances it is difficult to know whether a type I error or a type II error is being made or if no error is being made at all.

Question 41

What can we do in order to meet the assumption of the analysis of variance?

Answer 41

* In order to return our data to normality and establish homogeneity of variance we can use transformations. * These are simply mathematical operations that are applied to the data before we conduct an analysis of variance. * However, there are three circumstances where no transformation to the data will work: * Variances are heterogenous * Distributions are heterogenous * Variances are heterogeneous and distributions are heterogeneous

Question 42

In the case of skew the z-score is given by:

Answer 42

Z = skew-0 / SE skew

Question 43

The standard error of skew is given by:

Answer 43

SEskew = square root of 6/N * where N is the number of cases in the sample. * If a z score associated with the skew is greater than |±1.96| then the sample is significantly different from normal. * In other words, a value of skew which is significantly different from zero, would mean that we do not have normally distributed data