This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

How Do Their Results Compare?

Joseph C. Cappelleri, PhD, MPH; John P. A. loannidis, MD; Christopher H. Schmid, PhD; Sarah D. de Ferranti, MD, MPH; Michael Aubert; Thomas C. Chalmers, MD; Joseph Lau, MD

JAMA. 1996;276(16):1332-1338.

Abstract
Objective.
—To evaluate the results of large clinical trials vs the pooled results of smaller trials.

Data Identification.
—Mata-analyses with at least 1 "large" study were identified from the Cochrane Pregnancy and Childbirth Database and from MEDLINE (1966-1995).

Study Selection.
—We used a sample size approach to select 79 meta-analyses with at least 1 large study of 1000 or more patients. We used a statistical power approach to select 61 meta-analyses with at least 1 large study based on statistical power considerations.

Data Extraction.
—The outcome of interest for each meta-analysis was the primary one stated in the original publication or, when not clearly specified, was decided on clinically.

Data Synthesis.
—By random effects calculations, we found agreement between large and smaller trials in 90% of the meta-analyses selected by the sample size approach and in 82% of the meta-analyses selected by the statistical power approach. Twice as many disagreements appeared when the variability among large studies and among smaller studies was not considered (ie, fixed effects calculations). Of the 15 disagreements between results of large and smaller trials using the random effects model, plausible explanations were identified in 10 meta-analyses: 5 with differences in the control rate of events between large and smaller trials, 4 with specific protocol or study differences, and 1 with potential publication bias. Two other disagreements were not clinically important, and tentative reasons could be identified for 2 of the remaining 3 disagreements.

Conclusion.
—Results of smaller studies are usually compatible with the results of large studies, but discrepancies do occur even when the diversity among both large studies and smaller studies is considered. Clinically important differences without a potential explanation are extremely uncommon. Future research should further examine sources of heterogeneity between the results of large and smaller trials.

Author Affiliations
From the Division of Clinical Care Research (Drs Cappelleri, Schmid, de Ferranti, Chalmers, and Lau, and Mr Aubert), the Division of Geographic Medicine and Infectious Diseases (Dr loannidis), the Biostatistics Research Center (Dr Schmid), Department of Medicine, New England Medical Center, Boston, Mass. Dr Cappelleri is now with Pfizer Central Research, Groton, Conn; Dr loannidis is now with the National; Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Md; and Dr de Ferranti is now with Children's Hospital, Boston.

Footnotes
Deceased.

Reprints: Joseph Lau, MD, New England Medical Center, PO Box 63, 750 Washington St, Boston, MA 02111.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Critical Analysis of the Performance of Double-Contrast Barium Enema for Detecting Colorectal Polyps >= 6 mm in the Era of CT Colonography
Sosna et al.
Am. J. Roentgenol. 2008;190:374-385.
ABSTRACT | FULL TEXT  

The Avandia Debate
Bloomgarden
Diabetes Care 2007;30:2401-2408.
FULL TEXT  

How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania et al.
ANN INTERN MED 2007;147:224-233.
ABSTRACT | FULL TEXT  

Genetic Effects versus Bias for Candidate Polymorphisms in Myocardial Infarction: Case Study and Overview of Large-Scale Evidence
Ntzani et al.
Am J Epidemiol 2007;165:973-984.
ABSTRACT | FULL TEXT  

In Reply
Wagner et al.
JCO 2006;24:5474-5476.
FULL TEXT  

Meta-analysis: risk for hypertension in living kidney donors.
Boudville et al.
ANN INTERN MED 2006;145:185-196.
ABSTRACT | FULL TEXT  

Bias in Clinical Intervention Research
Gluud
Am J Epidemiol 2006;163:493-501.
ABSTRACT | FULL TEXT  

What the surgeon of tomorrow needs to know about evidence-based surgery.
Maier
Arch Surg 2006;141:317-323.
FULL TEXT  

Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou et al.
CMAJ 2006;174:635-641.
ABSTRACT | FULL TEXT  

Impact of Violations and Deviations in Hardy-Weinberg Equilibrium on Postulated Gene-Disease Associations
Trikalinos et al.
Am J Epidemiol 2006;163:300-309.
ABSTRACT | FULL TEXT  

Does Neostigmine Administration Produce a Clinically Important Increase in Postoperative Nausea and Vomiting?
Cheng et al.
Anesth. Analg. 2005;101:1349-1355.
ABSTRACT | FULL TEXT  

Effect of Initial Corticosteroid Therapy on Coronary Artery Aneurysm Formation in Kawasaki Disease: A Meta-analysis of 862 Children
Wooditch and Aronoff
Pediatrics 2005;116:989-995.
ABSTRACT | FULL TEXT  

Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis
JAMA 2005;294:218-228.
ABSTRACT | FULL TEXT  

Why industry should register and disclose results of clinical studies--perspective of a recovering academic
Berlin
BMJ 2005;330:959-959.
FULL TEXT  

Comparison of Large Versus Smaller Randomized Trials for Mental Health-Related Interventions
Contopoulos-Ioannidis et al.
Am. J. Psychiatry 2005;162:578-584.
ABSTRACT | FULL TEXT  

Evidence-based Colloid Use in the Critically Ill: American Thoracic Society Consensus Statement
Am. J. Respir. Crit. Care Med. 2004;170:1247-1259.
FULL TEXT  

Editorial II: Does epidural analgesia improve surgical outcome?
Ballantyne
Br J Anaesth 2004;92:4-6.
FULL TEXT  

Vagaries of genetic association studies in myocardial infarction
Mannucci et al.
Blood 2003;102:1558-1560.
FULL TEXT  

Appraising Systematic Reviews and Meta-analyses
Bigby and Williams
Arch Dermatol 2003;139:795-798.
ABSTRACT | FULL TEXT  

Using the Expected Survival to Explain Differences Between the Results of Randomized Trials: A Case in Advanced Ovarian Cancer
Buyse et al.
JCO 2003;21:1682-1687.
ABSTRACT | FULL TEXT  

Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses
Song et al.
BMJ 2003;326:472-472.
ABSTRACT | FULL TEXT  

Reported Methodologic Quality and Discrepancies between Large and Small Randomized Trials in Meta-Analyses
Kjaergard et al.
ANN INTERN MED 2001;135:982-989.
ABSTRACT | FULL TEXT  

Meta-analysis of randomized controlled trials on the comparative efficacy and safety of azithromycin against other antibiotics for lower respiratory tract infections
Contopoulos-Ioannidis et al.
J Antimicrob Chemother 2001;48:691-703.
ABSTRACT | FULL TEXT  

Role of p53 as a prognostic factor for survival in lung cancer: a systematic review of the literature with a meta-analysis
Steels et al.
Eur Respir J 2001;18:705-719.
ABSTRACT | FULL TEXT  

The Usefulness of Meta-Analyses in Treatment Decisions
Shrier et al.
JBJS 2001;83:1274-1275.
FULL TEXT  

Methods for Exploring Heterogeneity in Meta-Analysis
Song et al.
Eval Health Prof 2001;24:126-151.
ABSTRACT  

Safety Outcomes in Meta-analyses of Phase 2 vs Phase 3 Randomized Trials: Intracranial Hemorrhage in Trials of Bolus Thrombolytic Therapy
Eikelboom et al.
JAMA 2001;285:444-450.
ABSTRACT | FULL TEXT  

US Characterization of Ovarian Masses: A Meta-Analysis
Kinkel et al.
Radiology 2000;217:803-811.
ABSTRACT | FULL TEXT  

Prediction of Deep Myometrial Invasion in Patients with Endometrial Cancer: Clinical Utility of Contrast-enhanced MR Imaging-A Meta-analysis and Bayesian Analysis
Frei et al.
Radiology 2000;216:444-449.
ABSTRACT | FULL TEXT  

The Effect of Inhaled Steroids on the Linear Growth of Children With Asthma: A Meta-analysis
Sharek and Bergman
Pediatrics 2000;106:8e-8.
ABSTRACT | FULL TEXT  

Systemic Corticosteroids in Infant Bronchiolitis: A Meta-analysis
Garrison et al.
Pediatrics 2000;105:44e-44.
ABSTRACT | FULL TEXT  

Clinical Outcomes After Ablation and Pacing Therapy for Atrial Fibrillation : A Meta-Analysis
Wood et al.
Circulation 2000;101:1138-1144.
ABSTRACT | FULL TEXT  

ACE Gene Polymorphism in Cardiovascular Disease : Meta-Analyses of Small and Large Studies in Whites
Agerholm-Larsen et al.
Arterioscler. Thromb. Vasc. Bio. 2000;20:484-492.
ABSTRACT | FULL TEXT  

A Meta-Analysis of the Effectiveness of Cell Salvage to Minimize Perioperative Allogeneic Blood Transfusion in Cardiac and Orthopedic Surgery
Huet et al.
Anesth. Analg. 1999;89:861-861.
ABSTRACT | FULL TEXT  

The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Juni et al.
JAMA 1999;282:1054-1060.
ABSTRACT | FULL TEXT  

Radiologic Staging in Patients with Endometrial Cancer: A Meta-analysis
Kinkel et al.
Radiology 1999;212:711-718.
ABSTRACT | FULL TEXT  

Resolving Discrepancies Between a Meta-analysis and a Subsequent Large Controlled Trial
DerSimonian and Levine
JAMA 1999;282:664-670.
ABSTRACT | FULL TEXT  

Has the Use of Meta-Analysis Enhanced Our Understanding of Therapies for Postoperative Nausea and Vomiting?
White and Watcha
Anesth. Analg. 1999;88:1200-1200.
FULL TEXT  

Comparative Efficacy and Safety of Ondansetron, Droperidol, and Metoclopramide for Preventing Postoperative Nausea and Vomiting: A Meta-Analysis
Domino et al.
Anesth. Analg. 1999;88:1370-1370.
ABSTRACT | FULL TEXT  

The Role of Meta-analysis in the Regulatory Process for Foods, Drugs, and Devices
Berlin and Colditz
JAMA 1999;281:830-834.
ABSTRACT | FULL TEXT  

Meta-analysis and Epidemiologic Studies in Drug Development and Postmarketing Surveillance
Temple
JAMA 1999;281:841-844.
FULL TEXT  

Evidence-Based Medicine in a Nutshell: A Guide to Finding and Using the Best Evidence in Caring for Patients
Bigby
Arch Dermatol 1998;134:1609-1618.
ABSTRACT | FULL TEXT  

Guides for Reading and Interpreting Systematic Reviews: III. How Did the Authors Synthesize the Data and Make Their Conclusions?
Moher et al.
Arch Pediatr Adolesc Med 1998;152:915-920.
FULL TEXT  

Comparing Results From Meta-analyses vs Large Trials
LeLorier et al.
JAMA 1998;280:518-519.
FULL TEXT  

The Reproducibility of a Method to Identify the Overuse and Underuse of Medical Procedures
Shekelle et al.
NEJM 1998;338:1888-1895.
ABSTRACT | FULL TEXT  

Issues in Comparisons Between Meta-analyses and Large Trials
Ioannidis et al.
JAMA 1998;279:1089-1093.
ABSTRACT | FULL TEXT  

Managing Attribute--Value Clinical Trials Data Using the ACT/DB Client--Server Database System
Nadkarni et al.
J. Am. Med. Inform. Assoc. 1998;5:139-151.
ABSTRACT | FULL TEXT  

Effect of the Statistical Significance of Results on the Time to Completion and Publication of Randomized Efficacy Trials
Ioannidis
JAMA 1998;279:281-286.
ABSTRACT | FULL TEXT  

Meta-Analyses and Large Randomized, Controlled Trials
Ioannidis et al.
NEJM 1998;338:59-62.
FULL TEXT  

Radiological Evaluation of Lymph Node Metastases in Patients With Cervical Cancer: A Meta-analysis
Scheidler et al.
JAMA 1997;278:1096-1101.
ABSTRACT  

Large Trials vs Meta-analysis of Smaller Trials
Johnson et al.
JAMA 1997;277:377-377.
ABSTRACT  

Large Trials vs Meta-analysis of Smaller Trials
Klebanoff et al.
JAMA 1997;277:376-377.
ABSTRACT  

Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses
Ioannidis and Lau
Proc. Natl. Acad. Sci. USA 2001;98:831-836.
ABSTRACT | FULL TEXT