Some Practical Guidelines for Effective Sample Size Determination
Causal Question / Estimand
n/a as a causal estimand — a study-design paper on choosing sample size so an experiment has adequate Statistical-Power to detect an effect of substantive size.
Identification Strategy
n/a — methodological/statistical. Lays out how to do a credible power/sample-size calculation: specify the test, the error variance(s), and a scientifically meaningful effect size a priori; then solve for the sample size delivering target power. Stresses that effect size and variance should come from substantive reasoning or pilot data, not be reverse-engineered to justify a desired n.
Key Assumptions
n/a — theoretical/statistical. Assumes a correctly specified test and honest, externally grounded inputs (effect size, variance).
Threats to Validity
n/a — theoretical contribution. Warns against bad practice: retrospective/observed power (computed from the realized p-value, which is uninformative), circular “canned” effect sizes, and treating sample size as the only design lever when reducing variance (better measurement, blocking) may matter more.
Setting / Data
n/a — methodological, with generic illustrative examples.
Key Claims
- A useful power analysis requires an externally justified effect size and variance, specified before data collection.
- Observed (retrospective) power is not a meaningful diagnostic.
- Sample size is not always the binding issue — design changes that cut variance can be more effective than simply increasing n.
Connections
- Foundational for Statistical-Power; complements the design choices in BruhnMcKenzie2009-InPursuitOfBalance.
- Underpowered studies are a credibility threat catalogued by ChristensenMiguel2018-TransparencyReproducibility. See also RCT
Citation
Lenth, R. V. (2001). Some Practical Guidelines for Effective Sample Size Determination. The American Statistician, 55(3), 187–193.