How do researchers design studies to prevent internal validity threats?
The 6 threats to one-group, pretest/posttest designs can be ruled out if an experimenter conducts the study using a comparison group (either a posttest-only design or a pretest/
posttest design).
How do you interrogate an experiment with a null result to decide whether the study design obscured an effect or if there is truly no relationship?
Obscuring factors can be sorted into two categories:
1. Not enough between-groups difference (from weak manipulations, insensitive measures, ceiling or floor effects, or a design confound acting in reverse.)
2. Too much within-groups variance (from measurement error, irrelevant indi-
vidual differences, or situation noise.)
Too much in-group variance can be counteracted by using multiple measurements, more precise measurements, within-groups designs, large samples, and very controlled experimental
environments.
What are the 6 threats to internal validity that are especially relevant to the one-group, pretest/posttest design?
- Maturation - where there is a change in behaviour that emerges spontaneously over time > can be mitigated via comparison group
- History - results from a historical or external factor that systematically affects most members of the treatment group at the same time as treatment itself > a comparison group could help
- Regression - when a group average is unusually extreme at time 1, the next time that group is measured is likely to be less extreme > comparison group
- Attrition - when a certain kind of participant drops out a lot (systematic) > remove participant data
- Testing - a specific kind of order effect, refers to a change in the participants as a result of taking a test more than once > using only posttest design
- Instrumentation - occurs when a measuring instrument changes over time > calibrate instruments and counterbalancing
What are the 3 potential internal validity threats to any experiment?
- Observer bias - occurs when researchers’ expectations influence their interpretation of the results
- Demand characteristics - when participants guess what the study is supposed to be about and change their behavior in the expected direction
- Placebo effects - occurs when people recieve a treatment and really improve but only because the recipients believe they are recieveing a valid treatment
What is the really bad experiment?
The one-group, pretest/posttest design is like a pretest/posttest design has no comparison groups
What are the supgroups of selection threats?
- Selection-history threat - an outside event or factor effects only those at one level of the independent variable
- Selection-attrition threat - only one of the experimental groups experiences attrition
