Simple Random Samples
all observations have an equal chance of being selected for the treatment group and control group.
Complete Random Samples
the treatment and control groups are randomly assigned and have equal sizes.
Blocked Random Samples
the treatment and control groups are randomly assigned, have equal sizes, and a sub-group like gender also has equal sizes
Convenience Samples
Cumming and Calin-Jageman (2024) define convenience samples as “practically-achievable samples from the population”, which Gerring and Christenson (2017) suggest are usually chosen for logistical reasons (e.g., accessible, cheap, easy to study)
Snowball Samples
Using one subject/respondent to find others, which is common in qualitative research involving interviews.
Reliability
Repeatability of the result
Validity
Measuring what you think that you are measuring
Estimators
The procedure that we use to obtain our numerical estimate
𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 = 𝐸𝑠𝑡𝑖𝑚𝑎𝑛𝑑 + 𝐵𝑖𝑎𝑠 + 𝑁𝑜𝑖𝑠𝑒
𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 = 𝐸𝑠𝑡𝑖𝑚𝑎𝑛𝑑 + 𝐵𝑖𝑎𝑠 + 𝑁𝑜𝑖𝑠𝑒
Estimate
the number that we get from our analysis
Estimand
the true population-based quantity of interest that we aim to learn
Bias
systematic error that is not correct on average
Noise
idiosyncratic error, often due to sampling variation
Standard Error
𝜎 ̄ 𝑋 = 𝜎/√𝑁
𝜎 ̄ 𝑋 = 𝜎/√𝑁 explained
𝜎 refers to the standard deviation, 𝑁 corresponds to the sample size, and ̄ 𝑋 is
the sample mean
Standard errors represent
- a measure of precision or uncertainty about the estimate in question
- the standard deviation of the sampling distribution
- how far the estimate is from the mean estimate
How does sample size impact standard error?
As the standard error decreases as the sample size increases
Margin of Error
𝑀 𝑂𝐸 = 𝑧 × 𝜎 ̄ 𝑋
Z Score
( (𝑥−𝜇)/𝜎)
Usually corresponds to one of these three critical values: 2.58, 1.96, or 1.64. Most often, though, the 𝑧 = 1.96, corresponding to a 95% confidence interval.
Confidence Interval
𝐶𝐼 = ̄ 𝑋 ± 𝑀 𝑂𝐸 = ̄ 𝑋 ± 1.96(z) × 𝜎 ̄ 𝑋
Null Hypothesis (𝐻0)
General statement or default position that the result occurred by chance–i.e., no relationship
Type I error (𝛼)
significance level/p-value:
∗ rejecting 𝐻0 when it is true (false positive)
∗ 𝛼/p-value = 1 - confidence level (see above)
· for example, 𝛼 = .05 for a 95% confidence level
Type II error (𝛽):
failing to reject 𝐻0 when it is false (false negative)
NHST
NHST is about making hypotheses/guesses that chance was not cause of the guess concerned. The whole point of NHST is to correctly reject the null hypothesis that the 90%/95%/99% confidence interval does not contain the chance version of the guess. 𝐻0 can be true or false, and your statistical test results R, ‘SPSS“, or whatever program you are using tell you whether or not to reject 𝐻0.
