What are hypothesis tests?
- tests one possible value (unlike how CI tests a range)
- determine the strength of evidence, provided by data, against the proposition the single value is the true mean
- work by comparing an estimate obtained from data with what we expect to find under H0
What is the research hypothesis?
The hypothesis that the research is designed to investigate.
What is the null hypothesis?
H0: hypothesis we test, typically a skeptical reaction to the research hypothesis.
What is the alternative hypothesis?
H1: specifies a departure from H0, typically corresponds to the research hypothesis.
What is a test-statistic?
Used to quantify how much our data-estimate differs from the value in our H0.
What does the t-test statstic show us?
t-stat will be small if H0 is true.
When comparing two means, what is the null hypothesis?
That the difference between the values is 0.
How is the p-value obtained?
Compare t-stat to a reference t-distribution, p-value is obtained.
What is the p-value?
The probability of observing a test statistic at least as extreme as the one observed, given H0 is true.
What p-values count as strong evidence against H0?
- 1 - no evidence
- 05 - weak evidence
- 01 - some evidence
- 001 - strong evidence
When do we reject H0?
When the p-value is small. We reject H0 in favour of H1.
What is the non-parametric alternative to the t-test?
The Mann-Whitney test.
What does non-parametric mean? What is another name for it?
Methods to deal with non-normal data.
“distribution-free”
How is the Mann-Whitney test done?
- data is converted into ranks:
H0: there is NO difference between the ranks of each group - if H0 true, average ranks of each group about equal
- if ranks are very different we have reason to believe that the samples were not drawn from the same population
What is the procedure for the Mann-Whitney test?
- combine the scores from both groups and rank them in order of increasing size
- take each group and calculate the sum (Wn, n is the group)
- find the test stat for each group (Un)
- choose either Un value as the test stat (Ustat), usually the smaller value is used
- the test stat is then compared to the relevant distribution
- U is approx normally distributed, standard value (z) can be found
- we can now compare this test stat to the distribution of the test stat under H0.
How is the standardised value for the Mann-Whitney test found?
z = (U - 0.5 - μU)/σU
What is statistical significance?
H0 is rejected, indicates that differences in group means are not likely due to sampling error.
What is practical significance?
Asks are the differences between samples big enough to have real meaning?
What is a type I error?
Conclude there is a genuine difference when there is in fact not (reject H0 when its true).
What is a type II error?
Conclude there is no difference when in reality there is (fail to reject H0 when it’s false).
What does ANOVA mean?
ANalysis Of VAriance
What is an ANOVA test?
One test that compares all means simultaneously.
- H1: at least one mean is different from at least one of the others
- H0 is true: expect group means to be similar and any differences to be from sampling variation alone
What type of statistic does an ANOVA test produce?
F-test statistic.
What is s^2B and how is it calculated (for F-test stat)?
- difference between group mean and overall mean
- mean sq between groups (ANOVA table)
s^2B = (Σ ni (x̄ i - x̄)^2)/k-1 = SSB/ k - 1
