In hypothesis testing, what do we always assume is true?
The null hypothesis
What is the null hypothesis?
This states there is no difference between the variables of interest
What value is used to calculate the likelihood or probability that the difference observed happened by chance?
The p value
What does a p-value of 0.02 signify?
That the probability your scenario happened by chance is only 2 in 100
When is the null hypothesis rejected?
When the p-value is below the significance threshold
What does it mean if the p-value is large/above significance threshold?
You fail to reject your null hypothesis therefore no evidence exists for the difference - it is likely due to chance
What is the commonly used cut-off for p-value? Why is this not a universal figure?
0.05
For studies such as GWAS, a much lower p-value is required
What is a type I error also known as and when does this happen?
This is a false positive and occurs when you reject the null hypothesis even though it is actually true
What is the frequency of having a type I error/false positive?
This is the same as the value you use for significance cut off
What is a type II error also known as and when does it occur?
This is a false negative and occurs when you fail to reject the null hypothesis even though it is actually false
What is type II error or false negative dependent on?
Sample size
The choice of statistical test used to determine your p-value depends on what three key factors?
- Study design (paired or independent)
- Outcome variable (continuous or categorical)
- Distribution (normal or non-normal)
how is a t-statistic calculated?
For independent data, it is calculated by taking the observed mean difference and dividing this by the standard error of difference between the means
What three assumptions does a t-test make?
- Data is continuous
- Data is normally distributed
- Variance in the two groups is equal (levene’s test)
What does levene’s test do and why is it important?
Levene’s test helps assess whether the variance between two groups is equal. This is used when interpreting t-test results:
- If levene’s test is >0.05 then we accept the null hypothesis and interpret the results relating to ‘equal variances assumed’
What are your options if the assumptions for a parametric are untrue?
- Transform the data
- Check the normality again. If ok - use a parametric test
- If not ok, use a non-parametric test
What transformations can you attempt if your data is:
- moderately positively skewed
- strongly positively skewed
- weakly positively skewed
- log transform (logx)
- reciprocal (1/x)
- square root (rootx)
What transformation method would you use if your data was:
- moderately negatively skewed
- strongly negatively skewed
- unequal variation
- square (x2)
- cube (x3)
- log/reciprocal/squareroot
What are the advantages of a non-parametric test? What are the disadvantages?
- make no assumption about underlying distribution of data
- less powerful than parametric
- difficult to get CIs
What is the non-parametric equivalent of a t-test?
Wilcoxon rank sum test or Mann-Whitney u test
Describe how a wilcoxon rank sum test works
- two independent groups: group1 and 2 where group1 is the smallest size group
- rank all observations into ascending order
- sum ranks for group 1 = test statistic T
- look up T on wilcoxon rank sum table of critical values to get P-value
What non-parametric is used for skewed data with more than two independent exposure groups? What is its parametric equivalent?
Kruskal-Wallis test
Parametric equivalent = ANOVA
What test is used to compare two binary categorical variables and obtain a p-value?
Chi squared test
What does the p-value of a chi-squared test tell us?
How likely the differences between our variables would have occurred by chance if there was truly no association
