 |
|
|
Vol. 280 No. 3, July 15, 1998 |
 |
|
|
|
|
|
|
|
|
|
Peer Review Congress |
|
|
|
US and Non-US Submissions
An Analysis of Reviewer Bias
Ann M. Link, MA
JAMA. 1998;280:246-247.
ABSTRACT
| |
Context.— Reviewers increasingly are asked to review manuscripts from outside their own country, but whether they are more likely to recommend acceptance of such manuscripts is not known.
Objective.— To assess whether US reviewers or non-US reviewers evaluate manuscripts differently, depending on whether the manuscripts are submitted from outside the United States or from the United States.
Design and Setting.— A retrospective analysis of all original submissions received by Gastroenterology in 1995 and 1996. Reviewers ranked manuscripts in 4 decision categories: accept, provisionally accept, reject with resubmission, or reject.
Main Outcome Measure.— Ranking of papers based on nationality of authors and reviewers.
Results.— The percentage of non-US manuscripts placed in each decision category by US (n=2355) and non-US reviewers (n=1297) was nearly identical (P=.31). However, US reviewers recommended acceptance of papers submitted by US authors more often than did non-US reviewers (P=.001). Non-US reviewers ranked US papers slightly more favorably than non-US papers (P=.09), while US reviewers ranked US papers much more favorably (P=.001).
Conclusions.— Reviewers from the United States and outside the United States evaluate non-US papers similarly and evaluate papers submitted by US authors more favorably, with US reviewers having a significant preference for US papers.
INTRODUCTION
MANY US journals cite increased numbers of international submissions.1-3 Seventy percent of submissions to Gastroenterology are from outside the United States. However, non-US submissions are less likely to be published based on peer review. These data prompted a study assessing whether US and non-US reviewers evaluate manuscripts submitted to Gastroenterology from outside the US differently than manuscripts submitted from US-based authors.
METHODS
Using a manuscript tracking database, a retrospective study was conducted of original research articles submitted to Gastroenterology during 1995 and 1996. Manuscripts not sent out for peer review and revised manuscripts were excluded. The corresponding author's nationality, each reviewer's nationality, and each reviewer's ranking of the paper were obtained. Reviewers who did not indicate a decision ranking were excluded. Decision rankings of papers were obtained from the reviewer forms. These forms ask reviewers to recommend one of the following decisions on a paper: accept, provisionally accept, reject with resubmission, or reject. Two nonblinded reviewers are asked to review each paper.
Authors and reviewers were identified as either "US" or "non-US." Reviews were tabulated according to author nationality, reviewer nationality, and reviewer ranking combinations. Comparisons of reviewer recommendations and reviewers' nationality were assessed using the  2 test for independence (with 3 df). Reviewer recommendations and origin of the paper were analyzed in the same manner.
RESULTS
International papers were assessed to determine if they were ranked differently by US and non-US reviewers. There were 2355 US reviewers and 1297 non-US reviewers of non-US papers. The percentages of international papers placed in each decision category by US and non-US reviewers were similar, indicating no significant correlation between a reviewers' nationality and the ranking of international papers (accept: 3.6% vs 3.2%; provisionally accept: 24.9% vs 24.7%; reject with resubmission: 26.8% vs 24.6%; reject: 44.7% vs 47.6%) (P=.31).
Next, US papers were analyzed to determine if there were differences in US and non-US reviewers' rankings. There were 1174 US reviewers and 449 non-US reviewers of US papers. US reviewers placed papers in the 3 acceptance categories more often than non-US reviewers (64% vs 55%) and also ranked US papers significantly higher (accept: 7.0% vs 3.6%; provisionally accept: 31.3% vs 30.5%; reject with resubmission: 26.1% vs 21.6%; reject: 35.6% vs 44.3%) (P=.001).
Data were analyzed to determine whether these findings indicated bias. Reviewers' evaluations were examined to determine whether US reviewers evaluated non-US and US papers similarly. This process was also used to determine whether non-US reviewers evaluate the 2 sets of papers similarly. Non-US reviewers rank papers submitted from the United States more favorably (accept: 3.6% vs 3.2%; provisionally accept: 30.5% vs 24.7%; reject with resubmission: 21.6% vs 24.6%; reject: 44.3% vs 47.6%; P=.09). Moreover, US reviewers show a stronger preference for US papers than non-US reviewers (accept: 7.0% vs 3.6%; provisionally accept: 31.3% vs 24.9%; reject with resubmission: 26.1% vs 26.8%; reject: 35.6% vs 44.7%; P=.001).
Based on logistic regression looking simultaneously at reviewer nationality and manuscript source, the data show that the manuscript source was significant (P=.001), with domestic papers having an odds ratio of 1.49 for background of review and background of acceptance, while the reviewer's nationality was not significant (P=.22) (Table 1).
|
|
|
|
Acceptance Rates Based on Dichotomization
|
|
|
COMMENT
Several factors prevent conclusive findings on international bias in this study. First, non-US reviewers were pooled because there were not enough authors and reviewers from the same country to obtain statistically significant results. Second, nationality was based on the corresponding author's location rather than actual nationality. Third, associate editors may choose non-US reveiwers who have similiar training and viewpoints; however, data regarding location of training were not available. Fourth, reviewers were not blinded. Most important, bias cannot be unequivocally inferred without measuring the quality of individual papers. Still, this study does provide insights into potential variability in manuscript evaluations across national lines and lends itself to further investigation and more complex studies.
AUTHOR INFORMATION
Presented at the Third International Congress on Peer Review in Biomedical Publication, Prague, Czech Republic, September 19, 1997.
I wish to extend a special thanks to WB Saunders for funding the presentation of this study at the Third International Congress on Peer Review in Biomedical Publication.
Reprints: Ann M. Link, MA, Journal of Immunology, 9650 Rockville Pike, Bethesda, MD 20814.
From Gastroenterology, Bethesda, Md. Ms Link is now with the Journal of Immunology, Bethesda, Md.
REFERENCES
1. Stossel TP, Stossel S. Declining American representation in leading clinical research journals. N Engl J Med. 1990;322:739-742.
ABSTRACT
2. LaRusso NF, Link AM. Gastroenterology: an international journal. Gastroenterology. 1995;108:625-626.
FULL TEXT
|
WEB OF SCIENCE
3. Berk PD. Hepatology and hepatology: the good, the bad, and the ugly. Hepatology. 1996;24:1530-1532.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
100 papers to read before you die
Evers
Hum Reprod 2009;0:dep336v1-dep336.
FULL TEXT
The effect of masking manuscripts for the peer-review process of an ophthalmic journal
Isenberg et al.
Br J Ophthalmol 2009;93:881-884.
ABSTRACT
| FULL TEXT
Nonscientific Factors Associated with Acceptance for Publication in The Journal of Bone and Joint Surgery (American Volume)
Okike et al.
JBJS 2008;90:2432-2437.
ABSTRACT
| FULL TEXT
The 100 Most Frequently Cited Articles in Ophthalmology Journals
Ohba et al.
Arch Ophthalmol 2007;125:952-960.
ABSTRACT
| FULL TEXT
Effect of Blinded Peer Review on Abstract Acceptance
Ross et al.
JAMA 2006;295:1675-1680.
ABSTRACT
| FULL TEXT
The Euro-Americanization of Race: Alien Perspective of African Americans vis-a-vis Trivialization of Skin Color
Hall
Journal of Black Studies 2005;36:116-128.
ABSTRACT
Did the US boycott of French products spread to include scientific output?
Begaud and Verdoux
BMJ 2004;329:1430-1431.
FULL TEXT
Peer Review at the American Journal of Roentgenology: How Reviewer and Manuscript Characteristics Affected Editorial Decisions on 196 Major Papers
Kliewer et al.
Am. J. Roentgenol. 2004;183:1545-1550.
ABSTRACT
| FULL TEXT
Factors Influencing the Publication of Randomized Controlled Trials in Child Health Research
Hartling et al.
Arch Pediatr Adolesc Med 2004;158:983-987.
ABSTRACT
| FULL TEXT
Judging the judges: the role of journal editors
Ray
QJM 2002;95:769-774.
FULL TEXT
The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias
Opthof et al.
Cardiovasc Res 2002;56:339-346.
FULL TEXT
Publication Bias in Editorial Decision Making
Olson et al.
JAMA 2002;287:2825-2828.
ABSTRACT
| FULL TEXT
Publications on paediatric anaesthesia: a quantitative analysis of publication activity and international recognition
Brambrink et al.
Br J Anaesth 2000;85:556-562.
ABSTRACT
| FULL TEXT
Which Countries Publish in Important Anesthesia and Critical Care Journals?
Boldt et al.
Anesth. Analg. 1999;88:1175-1180.
ABSTRACT
| FULL TEXT
Evidence on peer review---scientific quality control or smokescreen?
Goldbeck-Wood
BMJ 1999;318:44-45.
FULL TEXT
Peer Review in Prague
Rennie
JAMA 1998;280:214-215.
FULL TEXT
|
|
|
