Natural and Quasi-Experiments in Economics
Note: this note is built from an OCR extraction of a scanned PDF; spot-check exact equation numbers and quotations against the original before citing.
Causal Question / Estimand
How can studies that mimic randomized experiments — using variation from law changes, draft lotteries, and the like — credibly estimate treatment effects (Causal-Estimand) when treatment and comparison groups are not randomly assigned? A methodological guide to designing and judging such studies.
Identification Strategy
Imports the Campbell–Stanley quasi-experimental design language into economics. Treatment is identified by finding plausibly exogenous variation in an explanatory variable and a suitable comparison group, then differencing out fixed group and time effects — i.e. difference-in-differences. Emphasizes strengthening designs with multiple treatment/comparison groups and multiple pre/post periods to test the identifying assumptions.
Key Assumptions
Parallel-Trends (comparison group reproduces the treated group’s counterfactual path), exogeneity of the policy variation (a design-based stand-in for Randomization / Ignorability), and SUTVA (well-defined treatment, no cross-group contamination).
Threats to Validity
A catalogue of internal-validity threats (after Campbell): omitted trends, mean reversion / the Ashenfelter-Dip, contamination of the comparison group, endogenous policy timing, and specification of the functional form. Argues these are checkable with richer designs (extra groups, extra periods, placebo comparisons).
Setting / Data
n/a — methodological. Draws on canonical natural-experiment examples (minimum-wage, unemployment-insurance, and draft-lottery studies) to illustrate design choices.
Key Claims
- Natural/quasi-experiments trade some external validity for credible internal validity via transparent, plausibly exogenous variation.
- Good design — multiple comparison groups and multiple time periods — lets the analyst test rather than merely assert the identifying assumptions.
- Specification and inference choices (e.g. serial correlation, functional form) materially affect conclusions and must be reported.
Connections
- See also: AngristPischke2010-CredibilityRevolution, AngristKrueger1999-EmpiricalStrategiesLaborEconomics, DiD, Foundations
- Anticipates: the modern DiD robustness literature (Bertrand2004-HowMuchShouldWeTrustDiD, RothSantAnna2023-WhatsTrendingInDiD)
- Shares phenomenon with: AshenfelterCard1985-LongitudinalEarnings (the pre-program dip as a threat to DiD)
Citation
Meyer, B. D. (1995). Natural and Quasi-Experiments in Economics. Journal of Business & Economic Statistics, 13(2), 151–161. https://doi.org/10.2307/1392369