This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citing articles on ISI (2)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Journalology/ Peer Review/ Authorship  • Statistics and Research Methods  • Randomized Controlled Trial  • Alert me on articles by topic

	Since this article does not have an abstract, we have provided the first 150 words of the full text and any section headings.

To the Editor: Dr Ioannidis¹ notes that randomized clinical trials failed to be replicated 9 of 39 times whereas nonrandomized (epidemiological) studies failed to be replicated 5 out of 6 times. Pocock et al² have also noted high false-positive rates in epidemiological studies. A possibly unappreciated explanation for failure of replication is the multiple comparisons problem.³ Researchers typically ask many questions of the same data set and formulate hypotheses in multiple ways (eg, covariate adjustment vs no adjustment; percentage change from baseline vs raw score, parametric vs nonparametric test), resulting in a multiplicity of features to choose to emphasize in the publication. The most dramatic results will naturally be emphasized. Failure of replication is essentially the same problem as investing in a so-called hot stock, based on its 50% return last year, and expecting the same 50% this year. Multiplicity invites selection, which in turn leads to regression to the . . . [Full Text of this Article]

Peter Westfall, PhD
peter.westfall@ttu.edu
Texas Tech University
Lubbock

S. Stanley Young, PhD
National Institute for Statistical Sciences
Research Triangle Park, NC

RELATED ARTICLES

Contradictions in Highly Cited Clinical Research
Colin P. West
JAMA. 2005;294(21):2695.
EXTRACT | FULL TEXT  

Contradictions in Highly Cited Clinical Research
Paul S. Heckerling
JAMA. 2005;294(21):2695.
EXTRACT | FULL TEXT  

Contradictions in Highly Cited Clinical Research—Reply
John P. A. Ioannidis
JAMA. 2005;294(21):2696.
EXTRACT | FULL TEXT  

Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
John P. A. Ioannidis
JAMA. 2005;294(2):218-228.
ABSTRACT | FULL TEXT