What are two fundamental relationships in academic research?
- Main effects: IV DV
- Moderations: IV DV (Influenced by moderating variable)
What is meant with the main effect?
- Hypothesis must include all variables and direction of the relationship
What are the three conditions for causality?
- Relationship between X and Y (concomitant variation)
X and Y must be related - Time order
First X then Y - Elimination of other possible causal factors
All other possible causes held constant or controlled
What are the three components of experimental design?
- Independent variable is manipulated
- Dependent variable is measured
- Controlling for extraneous factors
Give three reasons we use multi-item scales instead of single-item scales?
- Minimize error Reliability
- Capture complex, heterogeneous nature of construct or characteristic Validity
- Reveal more precisely the difference among participants
What are latent constructs?
unobservable constructs
On what two criteria does multiple item measurement happen?
- Reliability: Results are closely related
- Validity: They are where they should be inside the scope
Give 6 concrete examples of p-hacking?
- Cherry picking out of multiple outcomes
- Flexible data cleaning / outlier removal
- Selective subgroup analysis
- Trying multiple model specifications
- Optional stopping / peeking at data
- Transformations and recoding
How should reliability be measured?
Reliability should be measured with Cronbach’s alpha, where an alpha of higher than 0,7 is acceptable
What is meant with p-hacking?
practice of trying multiple analyses, variable selections, data-processing choices, or stopping rules and selectively reporting only those that produce statistically significant p-values
Give three reasons p-hacking is a problem?
- Inflates false positive rate
- Produces irreproducible and misleading findings
- Wastes resources and misdirects follow-up research and policy
Give three ways to prevent / detect p-hacking?
- Preregistration of hypotheses, outcomes and analysis plan
- Report all measured outcomes, contrasts, and specifications
- Share data and code on Open Science Framework to allow replication and re-analysis
Give two benefits of probability sampling?
- Reduce risk of sampling bias
- Enhance internal and external validity
Give one major drawback of probability sampling?
not easily collected in practice
What is convenience sampling?
anyone who passes by can be selected. Researcher will use his judgment to identify who will be in the sample.
What are the three advantages of convenience sampling?
Fast
Free or inexpensive
Easy to collect large and diverse samples quickly
Give three drawbacks of convenience sampling?
- Sample may be unrepresentative
- Strong selection bias
- Participants may be similar to researcher
