Week Nine

Question 1

types of follow up tests

Answer 1

○ A priori - decided before to test specific hypotheses.
○ Post hoc - comparisons made after assessing F ratio.
○ Nature of hypothesis tells you which test to use.

Question 2

a priori tests

Answer 2

(using t tests or planned comparisons)

○ Seek to compare only the groups of interest

Question 3

post hoc

Answer 3

• If you cannot predict exactly which means will differ then you should do the overall ANOVA first to see if the IV has an effect, then
  ○ Post hoc comparisons
  ○ Seek to compare all groups to each other to explore differences
  ○ Less refined - more exploratory

Question 4

planned comparisons procedure (a priori)

Answer 4

• To do this, we weigh our group means.
  ○ We assign weights or contrast coefficients (c) to reflect the means (M) we wish to compare.
  
  • In an example with 4 groups, assign group 1 a value of 1, group 2 a value of -1 and groups 3 and 4 a value of 0
  • Weights and coefficients are the same
  • They are numbers we assign to groups to communicate to SPSS and ourselves which groups we wish to compare.

Question 5

assigning weights

Answer 5

• sum of all must be 0
• groups that are being compared must have equal but opposite co-efficient.
  
  • When comparing two groups use 1 and -1.
  • When comparing groups use all parts of the group as 1 or -1.
  • Try to assign the group with the higher mean a value of 1.
  • If you are comparing one group to two groups, the other value will need to be + or - 2.
  • Use whole numbers in weights.

Question 6

planned contrasts and t

Answer 6

• T^2 = F
  
  • Therefore, can essentially compute a t statistic.
  • Easier to run the F test though.

Question 7

assumptions for planned contrasts

Answer 7

• Subject to the same assumptions as the ANOVA.
  ○ Particularly homogeneity of variance as we use pooled error term.
  ○ Do not have to run again though.
  ○ SPSS accounts for homogeneity of variance by giving homogeneity assumed and not assumed.

Question 8

type 1 errors and comparisons

Answer 8

• The more tests we conduct, the greater the chance of a type I error.
  
  • Alpha needs to be balanced, therefore lower the alpha rate by dividing 0.05 by the number of comparisons.
    ○ This is called a Bonferroni Adjustments
    ○ Then assess the tests using the new a value as the cut off
  • Planned comparison error rate = alpha
  • The error rate per experiment (PE) s the total number of Type 1 errors we are likely to make in conducting
    ○ PE= a x number of tests

Question 9

post hoc comparisons

Answer 9

• Need to correct type 1 error to maintain acceptable experiment error rate.
  
  • If each comparison is set at 0.05 and there are 6 comparisons. EW error rate - 0.35 which is not acceptable.
  • LSD method: least significant method (does no adjustment, alpha rates are not adjusted).
    ○ Can divide 0.05 by the number of tests and compare significance again.
  • Bonferroni method will adjust for you.
  • Tukey still uses 0.05 for comparisons.
  • All tests but LSB use 0.05 cut off.
  • When doing a post hoc method, you need to report on ALL results.

Question 10

effect size

Answer 10

• A significant F tells us that there is a difference between means. The Iv is having an effect.
  
  • Planned contrast or post hoc tests tell use where there effect is.
  • It does not tell us how strong or important this effect is.
  • We need a statistic that summarises the strength of the treatment effect:
    ○ Eta squared (n^2)
    ○ Indicates the proportion of the total variability in the data accounted for by the effect of the IV.
  • ** need to calculate n2 manually so remember the formula.

Question 11

eta squared

Answer 11

n^2= t^2/(t^2+df) = SSbetween/SStotal
- result says that that % of the variability in errors is due to the manipulation of the IV. 
- ranges from 0 to 1. 
Cohen suggests; 
- 0.01= small effect 
0.06= medium 
0.14= large

Question 12

problems with eta squared

Answer 12

• descriptive not inferential so not the best indicator of effect size in population.
  tends to overestimate the effect of size in population.

Question 13

cohen’s d

Answer 13

eta does not give an effect size for follow up tests - Cohen’s d is useful to measure effect sizes for a comparison of two means.
○ A priori and post hoc

• do not report cohen’s as a minus, use absolute value.

Question 14

cohen’s d formula

Answer 14

= u1-u2/ pop SD
= M1-M2/sqrt MSwithin

0. 2= small
1. 5 = medium
2. 8= large

Question 15

power

Answer 15

• Power is the probability of finding a significant effect when one exists.
  
  • Power = 1 - beta.
  • Power is a quantitative index of sensitivity which tells us the probability that our experiment will detect this effect.
  • Ideally, power should be > 0.80.
  • Power is a design issue.

Question 16

power can be increased by

Answer 16

• raising the a level (more type 1 errors tho)- raising alpha decreased beta so increases power.
• reducing error variance (good design and measures).
• increasing sample size
• increasing the number of conditions or groups
  0 increasing the treatment effect size.

Question 17

power and sample size

Answer 17

• ideally, we would determine the sample size that would gie a power of >0.8 before we run it.
• this can be determined from past research, pilot study or an estimate of the minimum difference between the means that you consider relevant or important.

Question 18

when are we concerned about power?

Answer 18

• when we do not find a significant effect but there is evidence of a possible type 2 error.
• when planning a new experiment and wish to ensure adequate power to pick up the effect of the IV.