Gold Standard
Experimental research, often considered to be the ‘gold standard’ in research designs, is one of the most rigorous of all research designs
- Experiments are praised for their ability to isolate causal effects of X on Y and go beyond correlation
- Yet, it is not always possible to run experiments in the lab or in the field
What is a Quasi-Experiment in the Field?
When a “treatment” happens (quasi-randomly) in the field
Difference in Difference (DiD) Strategy
Essentially a within-group or subject comparison
Difference in Difference (DiD) Strategy
The control group basically tells us…
what would have happened to the treatment group, had the treatment group not gotten the treatment
Three essential assumptions to identify a causal treatment effect
- Common Trend Assumption (CTA)
- Stable Unit Treatment Value Assumption (SUTVA)
- Conditional Independence Assumption (CIA)
- Common Trend Assumption (CTA)
It looks like there is a treatment effect after the intervention
But, if the two groups do NOT move in parallel before the treatment, the effect you measure after the intervention could be by coincidence.
When both groups, however, move in parallel before the intervention, there is reason to believe that is was actually the intervention that had a causal effect
Stable Unit Treatment Value Assumption (SUTVA)
- The treatment status of any unit does not affect the potential outcomes of the other units (non-interference)
- The treatments for all units are comparable (no variation in treatment)
- Non-interference
- No variation in treatment
Conditional Independence Assumption (CIA)
After controlling for differences in X, participation in the treatment program does not depend on potential (or latent) outcomes Y.
𝛽 3 identifies the DiD effect
The average treatment effect on the treated (ATT)
(C-A)-(D-B) Difference in changes over time
The best way to test the common trend assumption is by…
having data for many time periods before and after the intervention. Then you basically implement many interactions of the treatment variable with the time dummies.
What about staggered introduction?
- In the scenario with one treatment that happens everywhere at the same time, one unobserved other factor that happens at the same time can be a threat to the identification of the causal effect
- DiD models can also be used, when you have a treatment that is introduced at different locations at different times
- This has the great advantage that biasing factors need to be correlated over time AND location to make your identification invalid
Fixed effects can be to answers if you are…
concerned that your panels differ with regards to variables that you cannot observe (culture, mentality, etc)
Fixed effects
- This is basically a dummy variable for every panel. Can be easily implemented in R using plm() or lfe() (instead of using lm())
- You can have multiple fixed effects in the same model (e.g. country-, city-, year- fixed effects)
What fixed effects effectively does
- Fixed effects averages the slopes of all the panels. In this way, it only (not meant in a negative way) leverages variation in Y within the panels
- That is why they are also called within models
We can use DiD strategy to…
analyze quasi-experimental data in the field
DiD: The credibility of the results always depend on…
whether the crucial assumptions hold. Every study deploying a DiD analysis should present a compelling evidence and reasoning for this.
