What are some things done to preserve internal validity?
- control groups
- standardized, controlled, environment
- many participants
- double-blind study design
Threats to internal validity
Design Confounds
There is a problem with how the experiment is designed
Threats to internal validity
Selection Effects
Levels of IV differ because the participants in each level are different
Threats to internal validity
Order Effects
- within groups design
- differences may be due to IV or the order that info is presented
- Includes fatigue and practice effects
Threats to internal validity
What are maturation threats
- A change in behavior that emerges spontaneously as a result of time, ex. spontaneous remission of depression
Threats to internal validity
How can you prevent maturation threats?
- include a comparison/control group
- Ex. Campers that do not have low sugar diet, or women enrolled in a different type of therapy
Threats to internal validity
What are history Threats
- Threat occurs because something specific has happened between the pretest and posttest that influences most participants in the same way
- Ex. weather changing
Threats to internal validity
How can you prevent history threats?
With a good comparison/control group
Threats to internal validity
What is regression threat?
- Due to regression to the mean: when a group average is extreme at time 1, it is often less extreme when the behavior is measured again at time 2. Due to beginner’s luck or choking
- Extreme scores are rare. You should expect that extreme measures on time one will regress to the mean. People who are chosen because they are extreme also regress to the mean at time 2.
Threats to internal validity
How to prevent regression threat?
Have a comparison group
Threats to internal validity
What is attrition threat
- Attrition: when participations drop out of the study before the end (common when pre and post test occur on different days)
- Attrition threat: when only certain kinds of participants drop out (the dropout isn’t random)
- Often in Clinical/medical studies. (Participants that live farther away from lab/clinic/hospital are less likely to come back, and are also more likely to live in food/health care deserts)
Threats to internal validity
How to prevent threats to attrition?
When people drop out, remove their scores from the entire data set
What is the difference between attrition and history threats
Attrition only affects some participants, history affects all
Threats to internal validity
What is testing threat?
- Specific kind of order effect
- Change in participant as a result of taking a test more than once.
- Participatns can get better at taking the test, or bored, or fatigued
Threats to internal validity
How to prevent test threats?
- Get rid of the pretest all together
- Use different forms for the 2 tests
- Have a comparison group
Threats to internal validity
Instrumentation threats
- Occurs when a measuring instrument changes over time
- Coders may change how they judge behaviors over time, or when a researcher uses a different form of measurement for pre/post test
Threats to internal validity
How to prevent instrumentation threats to internal validity?
- post test only design
- make sure pre and posttest measures are equivalent
- Retrain coders throughout the experiment
Threats to internal validity
Observer bias and demand characteristics
- Observer bias: When researchers’ expectations influence their interpretation of the results. Threatens internal and construct validity.
- Demand characteristics: Participants guess the purpose of the study and change their behavior
Threats to internal validity
How to control for observer bias and demand characteristics?
- Double-bind study design
- Masked design- the participants know and raters/coders do not
Threats to internal validity
Placebo effect
- When participant receives a treatment and measurably improves. But only bc of participant’s beliefs.
- Pain is very susceptible to placebo effects.
- Using a placebo equates expectancy
Threats to internal validity
How to control for placebo effect?
- Double blind placebo control study
- Neither the people treating nor the partcipants know which group they are in
- The type of placebo matters
Null effects overview
- When difference between levels of the IV are not big enough
- Sometimes null effects are important
Null Effects
Weak manipulations
- When IV is not altered enough to produce change in DV
- Ex. If I want to test does money=happiness. How much money is necessary to get the effect? $10 vs $10,000
Null effects
Insensitive measures
The change is too small to be detected
