Sampling-Based versus Design-Based Uncertainty in Regression Analysis
Causal Question / Estimand
What does a regression standard error actually quantify? The paper distinguishes uncertainty about a descriptive or causal estimand arising from sampling (the sample is a draw from a larger population) versus from assignment/design (which units are treated), and shows the two can differ.
Identification Strategy
Develops a framework where the population may be (nearly) fully observed, so the usual sampling-based view is the wrong source of randomness. Under design-based uncertainty, potential outcomes are fixed and randomness comes from treatment assignment; the appropriate variance — and hence the standard error — can be smaller than the conventional robust (sampling-based) one, because there is no uncertainty about units already in hand. Provides estimators that are valid for the design-based estimand.
Key Assumptions
A well-defined assignment mechanism; clarity about the target estimand (descriptive vs. causal) and the population (sample vs. super-population). Potential-Outcomes, SUTVA.
Threats to Validity
Mismatch between the estimand/population the researcher intends and the one the standard error implicitly assumes; using sampling-based robust SEs when the relevant uncertainty is design-based (or vice versa) gives misleading inference.
Setting / Data
n/a — econometric theory, with regression as the running example.
Key Claims
- Conventional robust standard errors implicitly assume sampling-based uncertainty; when the population is fully observed, they overstate uncertainty about causal/ descriptive estimands.
- The correct variance depends on whether the estimand is descriptive or causal and on the source of randomness.
- Provides standard errors valid under design-based uncertainty.
Connections
- Formalizes the Design-Based-Inference concept; underlies AtheyImbens2022-DesignBasedDiD and the randomization-inference view in AtheyImbens2017-EconometricsOfRandomizedExperiments.
- Foundations of inference: Foundations.
Citation
Abadie, A., Athey, S., Imbens, G. W., & Wooldridge, J. M. (2020). Sampling-Based versus Design-Based Uncertainty in Regression Analysis. Econometrica, 88(1), 265–296.