This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (92)  • Contact me when this article is cited  Related Content  • Similar articles in JAMA

Karin Huwiler-Müntener, MD; Peter Jüni, MD; Christoph Junker, MD,MSc; Matthias Egger, MD,MSc

JAMA. 2002;287:2801-2804.

Context  The evaluation of the methodologic quality of randomized controlled trials (RCTs) is central to evidence-based health care. Important methodologic detail may, however, be omitted from published reports, and the quality of reporting is therefore often used as a proxy measure for methodologic quality. We examined the relationship between reporting quality and methodologic quality of published RCTs.

Methods  Study of 60 reports of placebo-controlled trials published in English-language journals from 1985 to 1997. Reporting quality was measured using a 25-item scale based on the 1996 issue of the Consolidated Standards of Reporting Trials (CONSORT). Concealment of allocation, appropriate blinding, and analysis according to the intention-to-treat principle were indicators of methodologic quality. Methodologic quality was compared between groups of trials defined by reporting quality scores of low, intermediate, and high. Reporting quality scores were compared between groups defined by high and low methodologic quality.

Results  Among 23 trials of low reporting quality (median score, 9 [range, 3.5-10.5]), allocation concealment was unclear for all but 1 trial, but there were 16 trials (70%) with adequate blinding and 9 trials (39%) that had been analyzed according to the intention-to-treat principle. Among 18 trials of high reporting quality (median score, 18 [range 16.5-22.0]), there were 8 trials (44%) with adequate allocation concealment, 16 trials (89%) with adequate blinding, and 13 trials (72%) analyzed according to the intention-to-treat principle. The median reporting score was 15.0 for the 33 trials that were analyzed according to intention-to-treat principle and 14.5 for the 14 trials with on-treatment analyses (P = .67).

Conclusions  Similar quality of reporting may hide important differences in methodologic quality, and well-conducted trials may be reported badly. A clear distinction should be made between these 2 dimensions of the quality of RCTs.

Author Affiliations: Departments of Social and Preventive Medicine (Drs Huwiler-Müntener, Junker, and Egger) and Rheumatology and Clinical Immunology (Dr Jüni), University of Bern, Bern, Switzerland; and MRC Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol, England (Drs Jüni and Egger).

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Cognitive Behavior Therapy for Schizophrenia: Effect Sizes, Clinical Models, and Methodological Rigor
Wykes et al.
Schizophr Bull 2008;34:523-537.
ABSTRACT | FULL TEXT  

Quality of randomized clinical trials in juvenile idiopathic arthritis
Abrahamyan et al.
Rheumatology (Oxford) 2008;47:640-645.
ABSTRACT | FULL TEXT  

Effectiveness of Specialized Palliative Care: A Systematic Review
Zimmermann et al.
JAMA 2008;299:1698-1709.
ABSTRACT | FULL TEXT  

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study
Wood et al.
BMJ 2008;336:601-605.
ABSTRACT | FULL TEXT  

Improving the Quality of Abstract Reporting for Phase I Cancer Trials
Strevel et al.
Clin. Cancer Res. 2008;14:1782-1787.
ABSTRACT | FULL TEXT  

Scales to Assess the Quality of Randomized Controlled Trials: A Systematic Review
Olivo et al.
ptjournal 2008;88:156-175.
ABSTRACT | FULL TEXT  

Placebo-controlled trials of Chinese herbal medicine and conventional medicine comparative study
Shang et al.
Int J Epidemiol 2007;36:1086-1092.
ABSTRACT | FULL TEXT  

A Systematic Review of the Performance Characteristics of Clinical Event Monitor Signals Used to Detect Adverse Drug Events in the Hospital Setting
Handler et al.
J. Am. Med. Inform. Assoc. 2007;14:451-458.
ABSTRACT | FULL TEXT  

Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography
Sanderson et al.
Int J Epidemiol 2007;36:666-676.
ABSTRACT | FULL TEXT  

Assessing the Quality of Glucose Monitor Studies: A Critical Evaluation of Published Reports
Mahoney and Ellison
Clin. Chem. 2007;53:1122-1128.
ABSTRACT | FULL TEXT  

Improving the quality of reporting randomized controlled trials in cardiothoracic surgery: The way forward.
Tiruvoipati et al.
J. Thorac. Cardiovasc. Surg. 2006;132:233-240.
ABSTRACT | FULL TEXT  

Quality of reporting of randomized, controlled trials in cerebral palsy.
Anttila et al.
Pediatrics 2006;117:2222-2230.
ABSTRACT | FULL TEXT  

Reporting of randomized controlled trials in Hodgkin lymphoma in biomedical journals.
Kober et al.
JNCI J Natl Cancer Inst 2006;98:620-625.
ABSTRACT | FULL TEXT  

Quality of Randomized Controlled Trials Reporting in the Primary Treatment of Brain Tumors
Lai et al.
JCO 2006;24:1136-1144.
ABSTRACT | FULL TEXT  

A Systematic Review of Practice Standards and Research Ethics in Technology-Based Home Health Care Intervention Programs for Older Adults
Marziali et al.
J Aging Health 2005;17:679-696.
ABSTRACT  

Duration of Anticoagulation Following Venous Thromboembolism: A Meta-analysis
Ost et al.
JAMA 2005;294:706-715.
ABSTRACT | FULL TEXT  

Methodological Approach: American College of Chest Physicians Guidelines for the Prevention and Management of Postoperative Atrial Fibrillation After Cardiac Surgery
Fleisher et al.
Chest 2005;128:17S-23S.
ABSTRACT | FULL TEXT  

The Quality of Randomized Controlled Trials in Major Anesthesiology Journals
Greenfield et al.
Anesth. Analg. 2005;100:1759-1764.
ABSTRACT | FULL TEXT  

Trial protocols: time for more than tinkering
Penston
BMJ 2005;330:421-421.
FULL TEXT  

Commentary: Empirical evidence of attrition bias in clinical trials
Juni and Egger
Int J Epidemiol 2005;34:87-88.
FULL TEXT  

A Review of the True Methodological Quality of Nutritional Support Trials Conducted in the Critically Ill: Time for Improvement
Doig et al.
Anesth. Analg. 2005;100:527-533.
ABSTRACT | FULL TEXT  

Randomized Controlled Trials
Stolberg et al.
Am. J. Roentgenol. 2004;183:1539-1544.
FULL TEXT  

Quality of cancer care
Vardy and Tannock
Ann Oncol 2004;15:1001-1006.
ABSTRACT | FULL TEXT  

Quality of Abstracts Describing Randomized Trials in the Proceedings of American Society of Clinical Oncology Meetings: Guidelines for Improved Reporting
Krzyzanowska et al.
JCO 2004;22:1993-1999.
ABSTRACT | FULL TEXT  

Lessons for cluster randomized trials in the twenty-first century: a systematic review of trials in primary care
Eldridge et al.
Clin Trials 2004;1:80-90.
ABSTRACT  

The Number, Quality, and Coverage of Randomized Controlled Trials in Nephrology
Strippoli et al.
J. Am. Soc. Nephrol. 2004;15:411-419.
ABSTRACT | FULL TEXT  

Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
Soares et al.
BMJ 2004;328:22-24.
ABSTRACT | FULL TEXT  

Clinical trials in orthodontics II: assessment of the quality of reporting of clinical trials published in three orthodontic journals between 1989 and 1998
Harrison
J. Orthod. 2003;30:309-315.
ABSTRACT | FULL TEXT  

Methodological Differences in Clinical Trials Evaluating Nonpharmacological and Pharmacological Treatments of Hip and Knee Osteoarthritis
Boutron et al.
JAMA 2003;290:1062-1070.
ABSTRACT | FULL TEXT  

Association of Funding and Conclusions in Randomized Drug Trials: A Reflection of Treatment Effect or Adverse Events?
Als-Nielsen et al.
JAMA 2003;290:921-928.
ABSTRACT | FULL TEXT  

Analgesic Efficacy of Calcitonin for Vertebral Fracture Pain
Blau and Hoehns
The Annals of Pharmacotherapy 2003;37:564-570.
ABSTRACT | FULL TEXT  

The Need for and Characteristics of Randomized, Phase III Trials to Evaluate Symptom Management in Patients With Cancer
Varricchio and Sloan
JNCI J Natl Cancer Inst 2002;94:1184-1185.
FULL TEXT