Use of peer ratings to evaluate physician performance

P. G. Ramsey, M. D. Wenrich, J. D. Carline, T. S. Inui, E. B. Larson and J. P. LoGerfo
Department of Medicine, University of Washington School of Medicine, Seattle 98195.

OBJECTIVE--To assess the feasibility and measurement characteristics of ratings completed by professional associates to evaluate the performance of practicing physicians. DESIGN--The clinical performance of physicians was evaluated using written questionnaires mailed to professional associates (physicians and nurses). Physician-associates were randomly selected from lists provided by both the subjects and medical supervisors, and detailed information was collected concerning the professional and social relationships between the associate and the subject. Responses were analyzed to determine factors that affect ratings and measurement characteristics of peer ratings. SETTING AND PARTICIPANTS--Physician-subjects were selected from among practicing internists in New York, New Jersey, and Pennsylvania who received American Board of Internal Medicine certification 5 to 15 years previously. MAIN OUTCOME MEASURE--Physician performance as assessed by peers. RESULTS--Peer ratings are not biased substantially by the method of selection of the peers or the relationship between the rater and the subject. Factor analyses suggest a two-dimensional conceptualization of clinical skills: one factor represents cognitive and clinical management skills and the other factor represents humanistic qualities and management of psychosocial aspects of illness. Ratings from 11 peer physicians are needed to provide a reliable assessment in these two areas. CONCLUSIONS--These findings suggest that it is feasible to obtain assessments from professional associates of practicing physicians in areas such as clinical skills, humanistic qualities, and communication skills. Using a shorter version of the questionnaire used in this study, peer ratings provide a practical method to assess clinical performance in areas such as humanistic qualities and communication skills that are difficult to assess with other measures.

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Questionnaires for 360-degree assessment of consultant psychiatrists: development and psychometric properties
Lelliott et al.
Br. J. Psychiatry 2008;193:156-160.
ABSTRACT | FULL TEXT  

Association Between Maintenance of Certification Examination Scores and Quality of Care for Medicare Beneficiaries
Holmboe et al.
Arch Intern Med 2008;168:1396-1403.
ABSTRACT | FULL TEXT  

Assessing the professional performance of UK doctors: an evaluation of the utility of the General Medical Council patient and colleague questionnaires
Campbell et al.
Qual Saf Health Care 2008;17:187-193.
ABSTRACT | FULL TEXT  

Assessment of Radiology Physicians by a Regulatory Authority
Lockyer et al.
Radiology 2008;247:771-778.
ABSTRACT | FULL TEXT  

A critical analysis of mini peer assessment tool (mini-PAT)
Abdulla
JRSM 2008;101:22-26.
ABSTRACT | FULL TEXT  

Peer Evaluation in a Clinical Clerkship: Students' Attitudes, Experiences, and Correlations With Traditional Assessments
Levine et al.
Acad. Psychiatry 2007;31:19-24.
ABSTRACT | FULL TEXT  

Assessment in Medical Education
Epstein
NEJM 2007;356:387-396.
FULL TEXT  

Effect of Multisource Feedback on Resident Communication Skills and Professionalism: A Randomized Controlled Trial
Brinkman et al.
Arch Pediatr Adolesc Med 2007;161:44-49.
ABSTRACT | FULL TEXT  

Use of assessment to reinforce patient safety as a habit
Galbraith et al.
Qual Saf Health Care 2006;15:i30-i33.
ABSTRACT | FULL TEXT  

Evaluation of Resident Communication Skills and Professionalism: A Matter of Perspective?
Brinkman et al.
Pediatrics 2006;118:1371-1379.
ABSTRACT | FULL TEXT  

Assessment of Pediatricians by a Regulatory Authority
Violato et al.
Pediatrics 2006;117:796-802.
ABSTRACT | FULL TEXT  

A multi source feedback program for anesthesiologists: [Un programme de retroaction multisources pour les anesthesiologistes]
Lockyer et al.
Canadian J. Anesthesia 2006;53:33-39.
ABSTRACT | FULL TEXT  

A structured competency based training programme for junior trainees in emergency medicine: the "Dundee Model"
Cook et al.
Emerg. Med. J. 2006;23:18-22.
ABSTRACT | FULL TEXT  

Disciplinary Action by Medical Boards and Prior Behavior in Medical School
Papadakis et al.
NEJM 2005;353:2673-2682.
ABSTRACT | FULL TEXT  

A qualitative study of why general practitioners may participate in significant event analysis and educational peer assessment
Bowie et al.
Qual Saf Health Care 2005;14:185-189.
ABSTRACT | FULL TEXT  

Use of SPRAT for peer review of paediatricians in training
Archer et al.
BMJ 2005;330:1251-1253.
ABSTRACT | FULL TEXT  

How to assess your specialist registrar
Davies and Howells
Arch. Dis. Child. 2004;89:1089-1093.
FULL TEXT  

Review of instruments for peer assessment of physicians
Evans et al.
BMJ 2004;328:1240.
ABSTRACT | FULL TEXT  

Revalidation for general practitioners: randomised comparison of two revalidation models
Bruce et al.
BMJ 2004;328:687-691.
ABSTRACT | FULL TEXT  

Assessment of Competency in Child and Adolescent Psychiatry Training
Sargent et al.
Acad. Psychiatry 2004;28:18-26.
ABSTRACT | FULL TEXT  

360-degree assessment in a multidisciplinary team setting
Potter and Palmer
Rheumatology (Oxford) 2003;42:1404-1407.
ABSTRACT | FULL TEXT  

Physician Clinical Performance Assessment: Prospects and Barriers
Landon et al.
JAMA 2003;290:1183-1189.
ABSTRACT | FULL TEXT  

Multisource feedback: a method of assessing surgical practice
Violato et al.
BMJ 2003;326:546-548.
FULL TEXT  

Appraisal for consultant medical staff
Brown et al.
Adv. Psychiatr. Treat. 2003;9:152-158.
ABSTRACT | FULL TEXT  

Professionalism: Lifelong Commitment for Surgeons
Welling and Boberg
Arch Surg 2003;138:262-264.
ABSTRACT | FULL TEXT  

Personal development plans: making them work, making them count
Newby
Adv. Psychiatr. Treat. 2003;9:5-10.
ABSTRACT | FULL TEXT  

What Is an Academic General Internist?: Career Options and Training Pathways
Levinson and Linzer
JAMA 2002;288:2045-2048.
ABSTRACT | FULL TEXT  

Comparison of Faculty, Peer, Self, and Nurse Assessment of Obstetrics and Gynecology Residents
Davis
Obstet Gynecol 2002;99:647-651.
ABSTRACT | FULL TEXT  

Defining and Assessing Professional Competence
Epstein and Hundert
JAMA 2002;287:226-235.
ABSTRACT | FULL TEXT  

Issues of Concern for the Aging Anesthesiologist
Katz
Anesth. Analg. 2001;92:1487-1492.
FULL TEXT  

GMC no longer favours folder of evidence for revalidation
Brearley
BMJ 2001;322:792-792.
FULL TEXT  

GMC's proposals for revalidation
Newble et al.
BMJ 2001;322:358-358.
FULL TEXT  

GMC's proposals for revalidation would not be accurate, economical, or fair
Wakeford
BMJ 2000;321:1220-1220.
FULL TEXT  

Recertification in Internal Medicine: A Program of Continuous Professional Development
Wasserman et al.
ANN INTERN MED 2000;133:202-208.
ABSTRACT | FULL TEXT  

Revalidation: Revalidation in the United Kingdom: general principles based on experience in general practice
Southgate and Pringle
BMJ 1999;319:1180-1183.
FULL TEXT  

Mindful Practice
Epstein
JAMA 1999;282:833-839.
ABSTRACT | FULL TEXT  

Assessment of physician performance in Alberta: the Physician Achievement Review
Hall et al.
CMAJ 1999;161:52-57.
ABSTRACT | FULL TEXT  

Otolaryngologists Older Than 60 Years: Results of and Reflections on Survey Responses From 865 Colleagues Regarding Retirement
Ward and Pratt
Arch Otolaryngol Head Neck Surg 1999;125:263-268.
FULL TEXT  

Developing a Cooperative Provider-Based Statewide Peer Review Service: Early Experience
Helms and Wakefield
American Journal of Medical Quality 1996;11:46-50.
ABSTRACT