This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (134)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Statistics and Research Methods  • Review  • Randomized Controlled Trial  • Alert me on articles by topic  Social Bookmarking   What's this?

A Systematic Review

Victor M. Montori, MD, MSc; P. J. Devereaux, MD; Neill K. J. Adhikari, MD; Karen E. A. Burns, MD; Christoph H. Eggert, MD; Matthias Briel, MD; Christina Lacchetti, MHSc; Teresa W. Leung, BHSc; Elizabeth Darling, RM, BHSc; Dianne M. Bryant, PhD; Heiner C. Bucher, MD, MPH; Holger J. Schünemann, MD, PhD; Maureen O. Meade, MD, MSc; Deborah J. Cook, MD, MSc; Patricia J. Erwin, MLS; Amit Sood, MD; Richa Sood, MD; Benjamin Lo, MD; Carly A. Thompson, BHSc; Qi Zhou, PhD; Edward Mills, PhD; Gordon H. Guyatt, MD, MSc

JAMA. 2005;294:2203-2209.

Context  Randomized clinical trials (RCTs) that stop earlier than planned because of apparent benefit often receive great attention and affect clinical practice. Their prevalence, the magnitude and plausibility of their treatment effects, and the extent to which they report information about how investigators decided to stop early are, however, unknown.

Objective  To evaluate the epidemiology and reporting quality of RCTs involving interventions stopped early for benefit.

Data Sources  Systematic review up to November 2004 of MEDLINE, EMBASE, Current Contents, and full-text journal content databases to identify RCTs stopped early for benefit.

Study Selection  Randomized clinical trials of any intervention reported as having stopped early because of results favoring the intervention. There were no exclusion criteria.

Data Extraction  Twelve reviewers working independently and in duplicate abstracted data on content area and type of intervention tested, reporting of funding, type of end point driving study termination, treatment effect, length of follow-up, estimated sample size and total sample studied, role of a data and safety monitoring board in stopping the study, number of interim analyses planned and conducted, and existence and type of monitoring methods, statistical boundaries, and adjustment procedures for interim analyses and early stopping.

Data Synthesis  Of 143 RCTs stopped early for benefit, the majority (92) were published in 5 high-impact medical journals. Typically, these were industry-funded drug trials in cardiology, cancer, and human immunodeficiency virus/AIDS. The proportion of all RCTs published in high-impact journals that were stopped early for benefit increased from 0.5% in 1990-1994 to 1.2% in 2000-2004 (P<.001 for trend). On average, RCTs recruited 63% (SD, 25%) of the planned sample and stopped after a median of 13 (interquartile range [IQR], 3-25) months of follow-up, 1 interim analysis, and when a median of 66 (IQR, 23-195) patients had experienced the end point driving study termination (event). The median risk ratio among truncated RCTs was 0.53 (IQR, 0.28-0.66). One hundred thirty-five (94%) of the 143 RCTs did not report at least 1 of the following: the planned sample size (n = 28), the interim analysis after which the trial was stopped (n = 45), whether a stopping rule informed the decision (n = 48), or an adjusted analysis accounting for interim monitoring and truncation (n = 129). Trials with fewer events yielded greater treatment effects (odds ratio, 28; 95% confidence interval, 11-73).

Conclusions  RCTs stopped early for benefit are becoming more common, often fail to adequately report relevant information about the decision to stop early, and show implausibly large treatment effects, particularly when the number of events is small. These findings suggest clinicians should view the results of such trials with skepticism.

Author Affiliations: Department of Clinical Epidemiology and Biostatistics (Drs Montori, Devereaux, Adhikari, Burns, Bryant, Schünemann, Meade, Cook, Zhou, Mills, and Guyatt and Ms Lacchetti and Darling), Bachelor of Health Sciences Programme (Mss Leung and Thompson), and Department of Medicine (Drs Devereaux, Mead, Cook, Lo, and Guyatt), McMaster University, Hamilton, Ontario; Department of Medicine, Mayo Clinic College of Medicine, Rochester, Minn (Drs Montori, Eggert, A. Sood, and R. Sood and Ms Erwin); Interdepartmental Division of Critical Care, University of Toronto, Toronto, Ontario (Drs Adhikari and Burns); Basel Institute for Clinical Epidemiology, University Hospital, Basel, Switzerland (Drs Briel and Bucher); and Departments of Medicine and Social and Preventive Medicine, University at Buffalo, Buffalo, NY and Division of Clinical Research Development and INFORMAtion Translation, Italian National Cancer Institute Regina Elena, Rome, Italy (Dr Schünemann).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

RELATED ARTICLES

When (Not) to Stop a Clinical Trial for Benefit
Stuart J. Pocock
JAMA. 2005;294(17):2228-2230.
EXTRACT | FULL TEXT  

Randomized Controlled Trials
Janet M. Torpy, Cassio Lynm, and Richard M. Glass
JAMA. 2005;294(17):2262.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
Liberati et al.
ANN INTERN MED 2009;151:W-65-W-94.
ABSTRACT | FULL TEXT  

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration
Liberati et al.
BMJ 2009;339:b2700-b2700.
ABSTRACT | FULL TEXT  

Glycemic Control in Type 2 Diabetes: Time for an Evidence-Based About-Face?
Montori and Fernandez-Balsells
ANN INTERN MED 2009;150:803-808.
ABSTRACT | FULL TEXT  

JUPITER: A Few Words of Caution
Vaccarino et al.
Circ Cardiovasc Qual Outcomes 2009;2:286-288.
FULL TEXT  

Rosuvastatin prevented major cardiovascular events in people with elevated C-reactive protein
Campbell-Scherer and Montori
Evid. Based Med. 2009;14:48-48.
FULL TEXT  

Stopping clinical trials early for benefit: impact on estimation
Freidlin and Korn
Clin Trials 2009;6:119-125.
ABSTRACT  

Stopping trials for efficacy: an almost unbiased view
Goodman
Clin Trials 2009;6:133-135.
 

Stopping or Reporting Early for Positive Results in Randomized Clinical Trials: The National Cancer Institute Cooperative Group Experience From 1990 to 2005
Korn et al.
JCO 2009;27:1712-1721.
ABSTRACT | FULL TEXT  

Early Stopping for Benefit in National Cancer Institute-Sponsored Randomized Phase III Trials: The System Is Working
Sargent
JCO 2009;27:1543-1544.
FULL TEXT  

Patients and the public deserve big changes in evaluation of drugs
Garattini and Chalmers
BMJ 2009;338:b1025-b1025.
FULL TEXT  

Rosuvastatin in Patients with Elevated C-Reactive Protein
Gibbons et al.
NEJM 2009;360:1038-1042.
FULL TEXT  

Enteral feeding regimens and necrotising enterocolitis in preterm infants: a multicentre case-control study
Henderson et al.
Arch. Dis. Child. Fetal Neonatal Ed. 2009;94:F120-F123.
ABSTRACT | FULL TEXT  

Rituximab Maintenance for the Treatment of Patients With Follicular Lymphoma: Systematic Review and Meta-analysis of Randomized Trials
Vidal et al.
JNCI J Natl Cancer Inst 2009;101:248-255.
ABSTRACT | FULL TEXT  

Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?
Thorlund et al.
Int J Epidemiol 2009;38:276-286.
ABSTRACT | FULL TEXT  

Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses
Brok et al.
Int J Epidemiol 2009;38:287-298.
ABSTRACT | FULL TEXT  

Tight Glucose Control in Critically Ill Adults--Reply
Finfer and Delaney
JAMA 2008;300:2727-2728.
FULL TEXT  

Dexamethasone and Risk of Nausea and Vomiting and Postoperative Bleeding After Tonsillectomy in Children: A Randomized Trial
Czarnetzki et al.
JAMA 2008;300:2621-2630.
ABSTRACT | FULL TEXT  

Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols
Chan et al.
BMJ 2008;337:a2299-a2299.
ABSTRACT | FULL TEXT  

Intensive glucose control did not prevent important complications in type 2 diabetes
Montori and Malaga
Evid. Based Med. 2008;13:168-169.
FULL TEXT  

Clinical outcomes in randomized trials of off- vs. on-pump coronary artery bypass surgery: systematic review with meta-analyses and trial sequential analyses
Moller et al.
Eur Heart J 2008;29:2601-2616.
ABSTRACT | FULL TEXT  

The effect of shortening and varus collapse of the femoral neck on function after fixation of intracapsular fracture of the hip: A MULTI-CENTRE COHORT STUDY
Zlowodzki et al.
J Bone Joint Surg Br 2008;90-B:1487-1494.
ABSTRACT | FULL TEXT  

Stopping a trial early in oncology: for patients or for industry?
Trotta et al.
Ann Oncol 2008;19:1347-1353.
ABSTRACT | FULL TEXT  

Stopping Trials for Benefit Can (Sometimes) Benefit Patients
Pater et al.
JCO 2008;26:2787-2788.
FULL TEXT  

Media influence on Herceptin subsidization in Australia: application of the rule of rescue?
MacKenzie et al.
JRSM 2008;101:305-312.
ABSTRACT | FULL TEXT  

Reply: Ethical recruitment of patients for PGS trial
Ankum et al.
Hum Reprod 2008;23:1472-1473.
FULL TEXT  

Trastuzumab in the Adjuvant Treatment of Early-Stage Breast Cancer: A Systematic Review and Meta-Analysis of Randomized Controlled Trials
Dahabreh et al.
The Oncologist 2008;13:620-630.
ABSTRACT | FULL TEXT  

Leaping to conclusions
McCartney
BMJ 2008;336:1213-1214.
FULL TEXT  

Randomized Trials in Hemodialysis Patients: Time to Step Up to the Plate
Tonelli
JAMA 2008;299:2205-2207.
FULL TEXT  

Selective serotonin reuptake inhibitors for unipolar depression: a systematic review of classic long-term randomized controlled trials
Deshauer et al.
CMAJ 2008;178:1293-1301.
ABSTRACT | FULL TEXT  

What is "quality of evidence" and why is it important to clinicians?
Guyatt et al.
BMJ 2008;336:995-998.
FULL TEXT  

When Should Data and Safety Monitoring Committees Share Interim Results in Cardiovascular Trials?
Borer et al.
JAMA 2008;299:1710-1712.
FULL TEXT  

Effect of Perioperative Insulin Infusion on Surgical Morbidity and Mortality: Systematic Review and Meta-analysis of Randomized Trials
Gandhi et al.
Mayo Clin Proc. 2008;83:418-430.
ABSTRACT | FULL TEXT  

Randomized Trials in Oncology Stopped Early for Benefit
Wilcox et al.
JCO 2008;26:18-19.
FULL TEXT  

Putting evidence into context: some advice for guideline writers
Dartnell et al.
Evid. Based Nurs. 2008;11:6-8.
FULL TEXT  

IVF with preimplantation genetic screening, a promising new treatment with unexpectedly negative health outcomes: the Hippocratic role of Data Monitoring Committees
Ankum et al.
Hum Reprod 2008;23:1-3.
ABSTRACT | FULL TEXT  

Sodium Bicarbonate is Associated with an Increased Incidence of Contrast Nephropathy: A Retrospective Cohort Study of 7977 Patients at Mayo Clinic
From et al.
CJASN 2008;3:10-18.
ABSTRACT | FULL TEXT  

Efficacy of Antioxidant Supplementation in Reducing Primary Cancer Incidence and Mortality: Systematic Review and Meta-analysis
Bardia et al.
Mayo Clin Proc. 2008;83:23-34.
ABSTRACT | FULL TEXT  

Cooling for Neonatal Hypoxic Ischemic Encephalopathy: Do We Have the Answer?
Kirpalani et al.
Pediatrics 2007;120:1126-1130.
FULL TEXT  

Putting evidence into context: some advice for guideline writers
Dartnell et al.
Evid. Based Med. 2007;12:130-132.
FULL TEXT  

Low-Dose Steroids in ARDS
Fremont and Rice
Chest 2007;132:1095-1095.
FULL TEXT  

An Update on the Diagnosis of Adrenal Insufficiency and the Use of Corticotherapy in Critical Illness
Thomas and Fraser
The Annals of Pharmacotherapy 2007;41:1456-1465.
ABSTRACT | FULL TEXT  

Ethical Issues in Stopping Randomized Trials Early Because of Apparent Benefit
Mueller et al.
ANN INTERN MED 2007;146:878-881.
ABSTRACT | FULL TEXT  

Stopping at Nothing? Some Dilemmas of Data Monitoring in Clinical Trials
Goodman
ANN INTERN MED 2007;146:882-887.
ABSTRACT | FULL TEXT  

Combination Therapy for Uncomplicated Falciparum Malaria in Ugandan Children: A Randomized Trial
Dorsey et al.
JAMA 2007;297:2210-2219.
ABSTRACT | FULL TEXT  

Rescue Angioplasty or Repeat Fibrinolysis After Failed Fibrinolytic Therapy for ST-Segment Myocardial Infarction: A Meta-Analysis of Randomized Trials
Wijeysundera et al.
J Am Coll Cardiol 2007;49:422-430.
ABSTRACT | FULL TEXT  

Response to Letter Regarding Article, "Aspirin Use and Outcomes in a Community-Based Cohort of 7352 Patients Discharged After First Hospitalization for Heart Failure"
McAlister et al.
Circulation 2007;115:e55-e55.
FULL TEXT  

Current controversies in data monitoring for clinical trials
Pocock
Clin Trials 2006;3:513-521.
ABSTRACT  

A public health perspective on research ethics
Buchanan and Miller
J. Med. Ethics 2006;32:729-733.
ABSTRACT | FULL TEXT  

Preoperative intensive inspiratory muscle training to prevent postoperative pulmonary complications in high-risk patients undergoing CABG surgery: a randomized clinical trial.
Hulzebos et al.
JAMA 2006;296:1851-1857.
ABSTRACT | FULL TEXT  

Clinical trial design issues: at least 10 things you should look for in clinical trials.
Glasser and Howard
J Clin Pharmacol 2006;46:1106-1115.
ABSTRACT | FULL TEXT  

Survival with aromatase inhibitors and inactivators versus standard hormonal therapy in advanced breast cancer: meta-analysis.
Mauri et al.
JNCI J Natl Cancer Inst 2006;98:1285-1291.
ABSTRACT | FULL TEXT  

Have ALLHAT, ANBP2, ASCOT-BPLA, and So Forth Improved Our Knowledge About Better Hypertension Care?
Sawicki and McGauran
Hypertension 2006;48:1-7.
FULL TEXT  

Intensive insulin therapy in the medical ICU.
Hammer et al.
NEJM 2006;354:2069-2071.
FULL TEXT  

Clinical Trials in the Wake of Vioxx: Requiring Statistically Extreme Evidence of Benefit to Ensure the Safety of New Drugs
Roth-Cline
Circulation 2006;113:2253-2259.
FULL TEXT  

Update in critical care 2005.
Milbrandt et al.
Am. J. Respir. Crit. Care Med. 2006;173:833-841.
FULL TEXT  

The cost of life: should it matter to doctors?
Gianni
Ann Oncol 2006;17:357-358.
FULL TEXT  

More drug trials end early hiding potential problems
BMJ 2005;331:1390-1390.
FULL TEXT  

What's new in the other general journals
Tonks
BMJ 2005;331:1105-1106.
FULL TEXT  

When (Not) to Stop a Clinical Trial for Benefit
Pocock
JAMA 2005;294:2228-2230.
FULL TEXT