Mechanism Experiments and Policy Evaluations
Causal Question / Estimand
Argues for a distinct object of study: the effect of a hypothesized causal mechanism, estimated by a “mechanism experiment,” as opposed to the effect of a full policy estimated by a conventional policy evaluation.
Identification Strategy
Both rest on Randomization for internal validity; the contribution is conceptual. A policy evaluation randomizes an actual program and identifies its overall effect. A mechanism experiment instead randomizes a targeted intervention that manipulates a specific channel a policy is presumed to work through — testing the mechanism directly, often more cheaply and before scaling a policy. This sharpens external validity: by isolating a mechanism, results can speak to many policies that share it, addressing the common critique that a single RCT’s LATE-like estimate does not generalize.
Key Assumptions
Randomization, SUTVA. External-validity reasoning assumes the manipulated mechanism is the same one the target policy would operate through.
Threats to Validity
The mechanism studied may not be the one a real policy activates; mechanism experiments trade some policy realism for mechanism clarity; standard RCT threats (noncompliance, spillovers) still apply.
Setting / Data
n/a — methodological/expository (JEP), with policy examples (education, behavioral interventions).
Key Claims
- Experiments can target mechanisms, not just whole policies, often at lower cost and with broader relevance.
- Mechanism experiments improve external validity by identifying a channel that many policies share.
- They complement, rather than replace, policy evaluations in an evidence-based policy pipeline.
Connections
- Mechanism-via-design counterpart to assumption-heavy Mediation analysis (Keele2015-CausalMediationAnalysis, VanderWeele2016-MediationAnalysis): experiments to test mechanisms vs. modelling to decompose them.
- Builds on RCT foundations: AtheyImbens2017-EconometricsOfRandomizedExperiments, RCT. External-validity theme shared with IV/LATE (Angrist2022-EmpiricalStrategiesIlluminatingPath). See also Foundations.
Citation
Ludwig, J., Kling, J. R., & Mullainathan, S. (2011). Mechanism Experiments and Policy Evaluations. Journal of Economic Perspectives, 25(3), 17–38.