Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends
Causal Question / Estimand
The dynamic DiD/event-study treatment effect (Causal-Estimand), and specifically what happens to estimation and inference once researchers condition on having passed a pre-trends test — the near-universal practice of eyeballing an event-study plot for flat pre-treatment coefficients.
Identification Strategy
Not a new estimator — a critique of pre-testing. Analyzes the statistical consequences of the common workflow: test for pre-existing trends, then proceed only if the test “passes.” Shows (i) conventional pre-trends tests often have low power to detect the very trends that would bias the estimate, and (ii) conditioning on passing the pre-test distorts the sampling distribution — exacerbating bias and worsening confidence-interval coverage (a selection/pre-test bias).
Key Assumptions
Operates within the standard Parallel-Trends event-study setup; the point is that testing parallel trends does not rescue identification and can make inference worse. No-Anticipation for the pre-period coefficients.
Threats to Validity
n/a — the paper analyzes a threat (pre-test bias). Its recommendations: report power against economically relevant trend violations, use uniform/robust inference, and prefer sensitivity analysis over binary pre-tests.
Setting / Data
Theory plus Monte Carlo simulations calibrated to a survey of recent papers (70 event-study plots in three leading journals, 2014–2018) to show the issues bite in practice.
Key Claims
- Pre-trends tests are frequently underpowered; “passing” is weak evidence for parallel trends.
- Conditioning on the pre-test induces selection bias in the reported estimates and confidence intervals.
- Motivates partial-identification / sensitivity approaches over reflexive pre-testing.
Connections
- Motivates: RambachanRoth2023-MoreCredibleParallelTrends (sensitivity analysis as the constructive response)
- Alternative constructive response: FreyaldenhovenEtAl2019-PreEventTrendsPanelEventStudy (an IV-based repair that sidesteps pre-testing entirely; its pre-test-then-estimate simulations echo this paper’s warning)
- Related diagnosis of spurious pre-trends: SunAbraham2021-EventStudies
- Informal precursor: KahnLangLang2020-PromiseAndPitfallsOfDiD (pre-testing yields wrong standard errors; accepting the null is a Type-II trap)
- Synthesized in: RothSantAnna2023-WhatsTrendingInDiD
- See also: DiD
Citation
Roth, J. (2022). Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends. American Economic Review: Insights, 4(3), 305–322. https://doi.org/10.1257/aeri.20210236