This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

Effects of Referee Characteristics and Publication Language

Magne Nylenna, MD; Povl Riis, MD; Yngve Karlsson

JAMA. 1994;272(2):149-151.

Abstract
Objective.
—To study the association between referee characteristics and their manuscript assessments, the influence of manuscript language on referees' judgments, and the usefulness, quality, and extent of referees' free-text comments.

Design.
—Two nonauthentic, but realistic, short manuscripts with a number of common methodological flaws were sent to 180 Scandinavian referees. Through randomization, each referee received one of the manuscripts in English and the other manuscript in the national language. A structured assessment of the manuscript quality was expressed on a 5-point scale, and the impact of referee characteristics (age, gender, experience, and so on) was analyzed by multiple linear regression.

Main Outcome.
—Manuscript quality assessed by referees.

Results.
—A total of 312 reviews from 156 referees could be used for the study of referee characteristics and language. With increasing experience, the referees gave lower quality scores (P<.05). A tendency toward stricter assessment with younger age was seen (P<.05). No influence of referees' gender, specialty, or nationality was found. For the test manuscript of the poorest quality, the English version was assessed to be better than the national-language version (P<.05). A total of 159 of 312 reviews included free-text comments applicable for the methodological study. In 54 reviews (34%), no methodological comments accompanied the assessment, and in six reviews they were only incomplete. Wrong sampling unit was mentioned by one fourth of 80 referees. Only one referee mentioned the incorrect use of a parametric test in the analysis of data whose distribution was nonparametric.

Conclusions.
—Experienced and young referees gave a stricter assessment of the manuscripts than their less experienced and older colleagues. An English version seemed to be accepted more easily than a national-language version of the same manuscript. Most referees spontaneously mentioned the shortcomings of the manuscripts only as part of their overall judgment.

(JAMA. 1994;272:149-151)

Author Affiliations
From the Journal of the Norwegian Medical Association, Lysaker, Norway (Dr Nylenna); Medical Department C, Herlev (Denmark) University Hospital (Dr Riis); and the Journal of the Swedish Medical Association, Stockholm, Sweden (Mr Karlsson).

Footnotes
Presented at the Second International Congress on Peer Review in Biomedical Publication, Chicago, Ill, September 11, 1993.

Reprint requests to Journal of the Norwegian Medical Association, Fjellveien 5, N-1324 Lysaker, Norway (Dr Nylenna).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Is There Gender Bias in the Peer Review Process at Journal of Neurophysiology?
Lane and Linden
J. Neurophysiol. 2009;101:2195-2196.
FULL TEXT  

What errors do peer reviewers detect, and does training improve their ability to detect them?
Schroter et al.
JRSM 2008;101:507-514.
ABSTRACT | FULL TEXT  

Reviewing the Reviewers: Comparison of Review Quality and Reviewer Characteristics at the American Journal of Roentgenology
Kliewer et al.
Am. J. Roentgenol. 2005;184:1731-1735.
ABSTRACT | FULL TEXT  

Peer Review at the American Journal of Roentgenology: How Reviewer and Manuscript Characteristics Affected Editorial Decisions on 196 Major Papers
Kliewer et al.
Am. J. Roentgenol. 2004;183:1545-1550.
ABSTRACT | FULL TEXT  

Vita Contemplativa: Why I Stopped Trying to Understand the Real World
Starbuck
Organization Studies 2004;25:1233-1254.
ABSTRACT  

Peer Review: Integral to Science and Indispensable to Annals
Laine et al.
ANN INTERN MED 2003;139:1038-1040.
FULL TEXT  

Turning Lemons into Lemonade: Where Is the Value in Peer Reviews?
Starbuck
Journal of Management Inquiry 2003;12:344-351.
ABSTRACT  

Judging the judges: the role of journal editors
Ray
QJM 2002;95:769-774.
FULL TEXT  

Measuring the Quality of Editorial Peer Review
Jefferson et al.
JAMA 2002;287:2786-2790.
ABSTRACT | FULL TEXT  

Systematic reviews in epidemiology: why are we so far behind?
Dickersin
Int J Epidemiol 2002;31:6-12.
FULL TEXT  

Publications on paediatric anaesthesia: a quantitative analysis of publication activity and international recognition
Brambrink et al.
Br J Anaesth 2000;85:556-562.
ABSTRACT | FULL TEXT  

Reliability of Editors' Subjective Quality Ratings of Peer Reviews of Manuscripts
Callaham et al.
JAMA 1998;280:229-231.
ABSTRACT | FULL TEXT  

What Makes a Good Reviewer and a Good Review for a General Medical Journal?
Black et al.
JAMA 1998;280:231-233.
ABSTRACT | FULL TEXT  

Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports: A Randomized Controlled Trial
Godlee et al.
JAMA 1998;280:237-240.
ABSTRACT | FULL TEXT  

Unpublished Research From a Medical Specialty Meeting: Why Investigators Fail to Publish
Weber et al.
JAMA 1998;280:257-259.
ABSTRACT | FULL TEXT  

Peer Review for Journals as it Stands Today--Part 1
CAMPANARIO
Science Communication 1998;19:181-211.
ABSTRACT  

Meta-analysis: Unresolved issues and future developments
Smith and Egger
BMJ 1998;316:221-225.
FULL TEXT  

And now, evidence based editing
Smith and Rennie
BMJ 1995;311:826-826.
FULL TEXT  

JSGI: Where We Are I Year Later
Lobo
Reproductive Sciences 1995;2:515-515.