 |
|
Effect of Blinded Peer Review on Abstract Acceptance
Joseph S. Ross, MD;
Cary P. Gross, MD;
Mayur M. Desai, PhD, MPH;
Yuling Hong, MD, PhD;
Augustus O. Grant, MB, PhD;
Stephen R. Daniels, MD, PhD;
Vladimir C. Hachinski, MD, DSc;
Raymond J. Gibbons, MD;
Timothy J. Gardner, MD;
Harlan M. Krumholz, MD, SM
JAMA. 2006;295:1675-1680.
Context Peer review should evaluate the merit and quality of abstracts but may be biased by geographic location or institutional prestige. The effectiveness of blinded peer review at reducing bias is unknown.
Objective To evaluate the effect of blinded review on the association between abstract characteristics and likelihood of abstract acceptance at a national research meeting.
Design and Setting All abstracts submitted to the American Heart Association's annual Scientific Sessions research meeting from 2000-2004. Abstract review included the author's name and institution (open review) from 2000-2001, and this information was concealed (blinded review) from 2002-2004. Abstracts were categorized by country, primary language, institution prestige, author sex, and government and industry status.
Main Outcome Measure Likelihood of abstract acceptance during open and blinded review, by abstract characteristics.
Results The mean number of abstracts submitted each year for evaluation was 13 455 and 28.5% were accepted. During open review, 40.8% of US and 22.6% of non-US abstracts were accepted (relative risk [RR], 1.81; 95% confidence interval [CI], 1.75-1.88), whereas during blinded review, 33.4% of US and 23.7% of non-US abstracts were accepted (RR, 1.41; 95% CI, 1.37-1.45; P<.001 for comparison between peer review periods). Among non-US abstracts, during open review, 31.1% from English- speaking countries and 20.9% from non–English-speaking countries were accepted (RR, 1.49; 95% CI, 1.39-1.59), whereas during blinded review, 28.8% and 22.8% of abstracts were accepted, respectively (RR, 1.26; 95% CI, 1.19-1.34; P<.001). Among abstracts from US academic institutions, during open review, 51.3% from highly prestigious and 32.6% from nonprestigious institutions were accepted (RR, 1.57; 95% CI, 1.48-1.67), whereas during blinded review, 38.8% and 29.0% of abstracts were accepted, respectively (RR, 1.34; 95% CI, 1.26-1.41; P<.001).
Conclusions This study provides evidence of bias in the open review of abstracts, favoring authors from the United States, English-speaking countries outside the United States, and prestigious academic institutions. Moreover, blinded review at least partially reduced reviewer bias.
Author Affiliations: Robert Wood Johnson Clinical Scholars Program (Dr Ross), Department of Internal Medicine (Drs Ross, Gross, and Krumholz), Department of Psychiatry (Dr Desai), Section of Cardiovascular Medicine (Dr Krumholz), Section of Health Policy and Administration, and Department of Epidemiology and Public Health (Dr Krumholz), Yale University School of Medicine and Yale-New Haven Hospital Center for Outcomes Research and Evaluation (Dr Krumholz), New Haven, Conn; American Heart Association, Dallas, Tex (Dr Hong); Division of Cardiology, Department of Internal Medicine, Duke University Medical Center, Durham, NC (Dr Grant); Department of Pediatrics, Division of Cardiology, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio (Dr Daniels); Lawson Health Research Institute, Neurosciences Program, London, Ontario (Dr Hachinski); Division of Cardiovascular Diseases, Department of Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minn (Dr Gibbons); and Center for Heart & Vascular Health, Christiana Care Health System, Newark, Del (Dr Gardner). Dr Daniels is now with the Department of Pediatrics, University of Colorado, Denver.
RELATED LETTERS
Antiarrhythmic Therapy for Prevention of Implantable Cardioverter Defibrillator Shocks—Reply
Stefan H. Hohnloser, Robin S. Roberts, Eric Fain, and Stuart J. Connolly
JAMA. 2006;296(10):1230.
EXTRACT
| FULL TEXT
Abstract Acceptance and Blinded Peer Review
Matthew E. Falagas
JAMA. 2006;296(10):1230.
EXTRACT
| FULL TEXT
Abstract Acceptance and Blinded Peer Review—Reply
Joseph S. Ross and Harlan M. Krumholz
JAMA. 2006;296(10):1230-1231.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Epistemic Trust, Epistemic Responsibility, and Medical Practice
Schwab
J Med Philos 2008;33:302-320.
ABSTRACT
| FULL TEXT
Better reporting of randomized trials in biomedical journal and conference abstracts
Hopewell et al.
Journal of Information Science 2008;34:162-173.
ABSTRACT
Differences Among International Pharyngitis Guidelines: Not Just Academic
Matthys et al.
Ann Fam Med 2007;5:436-443.
ABSTRACT
| FULL TEXT
The ups and downs of peer review
Benos et al.
Adv. Physiol. Educ. 2007;31:145-152.
ABSTRACT
| FULL TEXT
Update in Critical Care 2006
Milbrandt et al.
Am. J. Respir. Crit. Care Med. 2007;175:638-648.
FULL TEXT
Abstract acceptance and blinded peer review.
Falagas
JAMA 2006;296:1230-1230.
FULL TEXT
|
