 |
|
Does Masking Author Identity Improve Peer Review Quality?
A Randomized Controlled Trial
Amy C. Justice, MD, PhD;
Mildred K. Cho, PhD;
Margaret A. Winker, MD;
Jesse A. Berlin, ScD;
Drummond Rennie, MD;
and the PEER Investigators
JAMA. 1998;280:240-242.
Context.— All authors may not be equal in the eyes of reviewers. Specifically, well-known authors may receive less objective (poorer quality) reviews. One study at a single journal found a small improvement in review quality when reviewers were masked to author identity.
Objectives.— To determine whether masking reviewers to author identity is generally associated with higher quality of review at biomedical journals, and to determine the success of routine masking techniques.
Design and Setting.— A randomized controlled trial performed on external reviews of manuscripts submitted to Annals of Emergency Medicine, Annals of Internal Medicine, JAMA, Obstetrics & Gynecology , and Ophthalmology .
Interventions.— Two peers reviewed each manuscript. In one study arm, both peer reviewers received the manuscript according to usual masking practice. In the other arm, one reviewer was randomized to receive a manuscript with author identity masked, and the other reviewer received an unmasked manuscript.
Main Outcome Measure.— Review quality on a 5-point Likert scale as judged by manuscript author and editor. A difference of 0.5 or greater was considered important.
Results.— A total of 118 manuscripts were randomized, 26 to usual practice and 92 to intervention. In the intervention arm, editor quality assessment was complete for 77 (84%) of 92 manuscripts. Author quality assessment was complete on 40 (54%) of 74 manuscripts. Authors and editors perceived no significant difference in quality between masked (mean difference, 0.1; 95% confidence interval [CI], -0.2 to 0.4) and unmasked (mean difference, -0.1; 95% CI, -0.5 to 0.4) reviews. We also found no difference in the degree to which the review influenced the editorial decision (mean difference, -0.1; 95% CI,-0.3 to 0.3). Masking was often unsuccessful (overall, 68% successfully masked; 95% CI, 58%-77%), although 1 journal had significantly better masking success than others (90% successfully masked; 95% CI, 73%-98%). Manuscripts by generally known authors were less likely to be successfully masked (odds ratio, 0.3; 95% CI, 0.1-0.8). When analysis was restricted to manuscripts that were successfully masked, review quality as assessed by editors and authors still did not differ.
Conclusions.— Masking reviewers to author identity as commonly practiced does not improve quality of reviews. Since manuscripts of well-known authors are more difficult to mask, and those manuscripts may be more likely to benefit from masking, the inability to mask reviewers to the identity of well-known authors may have contributed to the lack of effect.
From the Division of General Internal Medicine and Program for Health Care Research, Department of Veterans Affairs Medical Center and University Hospitals of Cleveland and Case Western Reserve University, Cleveland, Ohio (Dr Justice); the Center for Bioethics (Dr Cho) and the Division of Biostatistics, Department of Biostatistics and Epidemiology and Center for Clinical Epidemiology and Biostatistics, School of Medicine, University of Pennsylvania, Philadelphia (Dr Berlin); and the Institute for Health Policy Studies, University of California, San Francisco (Dr Rennie). Dr Winker is Senior Editor and Dr Rennie is Deputy Editor (West), JAMA.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
RELATED ARTICLE
July 15, 1998
JAMA. 1998;280(3):303-304.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
The effect of masking manuscripts for the peer-review process of an ophthalmic journal
Isenberg et al.
Br J Ophthalmol 2009;93:881-884.
ABSTRACT
| FULL TEXT
CORRECTION
Journal of the American Dental Association 2007;138:730-730.
FULL TEXT
Peer review: An inexact but essential part of scientific publishing
Glick
Journal of the American Dental Association 2007;138:568-571.
FULL TEXT
Comparison of treatment effects between animal experiments and clinical trials: systematic review
Perel et al.
BMJ 2007;334:197-197.
ABSTRACT
| FULL TEXT
How Masked Is the 'Masked Peer Review' of Abstracts Submitted to International Medical Conferences?
Falagas et al.
Mayo Clin Proc. 2006;81:705-705.
FULL TEXT
Peer review: a flawed process at the heart of science and journals.
Smith
JRSM 2006;99:178-182.
FULL TEXT
Necessary steps for a modern scientific journal.
Allebeck
Eur J Public Health 2006;16:1-1.
FULL TEXT
What makes the best medical ethics journal? A North American perspective
Savulescu and Viens
J. Med. Ethics 2005;31:591-597.
ABSTRACT
| FULL TEXT
Statin use after renal transplantation: a systematic quality review of trial-based evidence
Lentine and Brennan
Nephrol Dial Transplant 2004;19:2378-2386.
ABSTRACT
| FULL TEXT
Incidence and Nature of Unblinding by Authors: Our Experience at Two Radiology Journals with Double-Blinded Peer Review Policies
Katz et al.
Am. J. Roentgenol. 2002;179:1415-1417.
ABSTRACT
| FULL TEXT
Making Reviewers Visible: Openness, Accountability, and Credit
Godlee
JAMA 2002;287:2762-2765.
ABSTRACT
| FULL TEXT
Effects of Editorial Peer Review: A Systematic Review
Jefferson et al.
JAMA 2002;287:2784-2786.
ABSTRACT
| FULL TEXT
Measuring the Quality of Editorial Peer Review
Jefferson et al.
JAMA 2002;287:2786-2790.
ABSTRACT
| FULL TEXT
Peer review of statistics in medical research: the other problem
Bacchetti
BMJ 2002;324:1271-1273.
FULL TEXT
Referees' Attitudes toward Open Peer Review and Electronic Transmission of Papers
Melero and Lopez-Santovena
Food Science and Technology International 2001;7:521-527.
ABSTRACT
Reviewers, Authors, and Editors: Balance of Power
Bressler
Arch Ophthalmol 1999;117:524-526.
FULL TEXT
Opening up BMJ peer review
Smith
BMJ 1999;318:4-5.
FULL TEXT
Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial
van Rooyen et al.
BMJ 1999;318:23-27.
ABSTRACT
| FULL TEXT
Evidence on peer review---scientific quality control or smokescreen?
Goldbeck-Wood
BMJ 1999;318:44-45.
FULL TEXT
Peer Review in Prague
Rennie
JAMA 1998;280:214-215.
FULL TEXT
Masking Author Identity in Peer Review: What Factors Influence Masking Success?
Cho et al.
JAMA 1998;280:243-245.
ABSTRACT
| FULL TEXT
Freedom and Responsibility in Medical Publication: Setting the Balance Right
Rennie
JAMA 1998;280:300-302.
FULL TEXT
|
