Transparency, Reproducibility, and the Credibility of Economics Research
Causal Question / Estimand
Meta-scientific: not a treatment effect but the credibility of the body of empirical estimates — how research practices distort the published distribution of findings and what reforms restore trust.
Identification Strategy
Surveys the evidence that current practice produces biased bodies of evidence: publication bias (significant results selectively published), specification searching / p-hacking (researcher degrees of freedom), and underpowered studies (Statistical-Power) that, conditional on significance, exaggerate effects. Reviews the reform toolkit: study registration and [[Pre-Analysis-Plan|pre-analysis plans]], disclosure and reporting standards, replication/reproducibility, and open data and materials. The unifying logic is constraining the analytic flexibility that divorces published estimates from the true effect distribution.
Key Assumptions
n/a — review/meta-science, not an identified causal estimate. Implicitly invokes the Statistical-Power and selection mechanisms that bias literatures.
Threats to Validity
The paper catalogues the threats to the credibility of empirical work: selective reporting, publication bias, low power, non-reproducibility, and the incentives that sustain them.
Setting / Data
n/a — survey of economics (and adjacent social-science) research practices.
Key Claims
- Publication bias, specification search, and low power jointly make a substantial share of published findings unreliable.
- Pre-registration and pre-analysis plans, replication, and open data/disclosure are the core tools for restoring credibility.
- These reforms are the methodological complement to the design-based “credibility revolution” — credible designs still need credible reporting.
Connections
- Complements design-based identification: AngristPischke2010-CredibilityRevolution (credible designs) ↔ this (credible reporting).
- Pre-Analysis-Plan scope debated by DufloEtAl2020-InPraiseOfModeration.
- Power concerns shared with Lenth2001-SampleSizeDetermination, BruhnMcKenzie2009-InPursuitOfBalance. See also RCT
Citation
Christensen, G., & Miguel, E. (2018). Transparency, Reproducibility, and the Credibility of Economics Research. Journal of Economic Literature, 56(3), 920–980.