This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (146)  • Contact me when this article is cited  Related Content  • Related article  • Similar articles in JAMA  Topic Collections  • Medical Practice  • Medical Education  • Quality of Care  • Quality of Care, Other  • Review  • Alert me on articles by topic  Social Bookmarking   What's this?

A Systematic Review

David A. Davis, MD; Paul E. Mazmanian, PhD; Michael Fordis, MD; R. Van Harrison, PhD; Kevin E. Thorpe, MMath; Laure Perrier, MEd, MLIS

JAMA. 2006;296:1094-1102.

Context  Core physician activities of lifelong learning, continuing medical education credit, relicensure, specialty recertification, and clinical competence are linked to the abilities of physicians to assess their own learning needs and choose educational activities that meet these needs.

Objective  To determine how accurately physicians self-assess compared with external observations of their competence.

Data Sources  The electronic databases MEDLINE (1966-July 2006), EMBASE (1980-July 2006), CINAHL (1982-July 2006), PsycINFO (1967-July 2006), the Research and Development Resource Base in CME (1978-July 2006), and proprietary search engines were searched using terms related to self-directed learning, self-assessment, and self-reflection.

Study Selection  Studies were included if they compared physicians' self-rated assessments with external observations, used quantifiable and replicable measures, included a study population of at least 50% practicing physicians, residents, or similar health professionals, and were conducted in the United Kingdom, Canada, United States, Australia, or New Zealand. Studies were excluded if they were comparisons of self-reports, studies of medical students, assessed physician beliefs about patient status, described the development of self-assessment measures, or were self-assessment programs of specialty societies. Studies conducted in the context of an educational or quality improvement intervention were included only if comparative data were obtained before the intervention.

Data Extraction  Study population, content area and self-assessment domain of the study, methods used to measure the self-assessment of study participants and those used to measure their competence or performance, existence and use of statistical tests, study outcomes, and explanatory comparative data were extracted.

Data Synthesis  The search yielded 725 articles, of which 17 met all inclusion criteria. The studies included a wide range of domains, comparisons, measures, and methodological rigor. Of the 20 comparisons between self- and external assessment, 13 demonstrated little, no, or an inverse relationship and 7 demonstrated positive associations. A number of studies found the worst accuracy in self-assessment among physicians who were the least skilled and those who were the most confident. These results are consistent with those found in other professions.

Conclusions  While suboptimal in quality, the preponderance of evidence suggests that physicians have a limited ability to accurately self-assess. The processes currently used to undertake professional development and evaluate competence may need to focus more on external assessment.

Author Affiliations: Knowledge Translation Program of the Li Ka Shing Knowledge Institute at St Michael's Hospital (Dr Davis and Mr Thorpe), Departments of Health Policy, Management, and Evaluation (Dr Davis), Family and Community Medicine (Dr Davis), and Public Health Sciences (Mr Thorpe), and the Office of Continuing Education and Professional Development (Ms Perrier), University of Toronto, Toronto, Ontario; Departments of Family Medicine and Epidemiology and Community Health, School of Medicine, Virginia Commonwealth University, Richmond (Dr Mazmanian); Center for Collaborative and Interactive Technologies, Baylor College of Medicine, Houston, Tex (Dr Fordis); and Department of Medical Education, University of Michigan, Ann Arbor (Dr Harrison).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

RELATED ARTICLE

Self-assessment in Lifelong Learning and Improving Performance in Practice: Physician Know Thyself
F. Daniel Duffy and Eric S. Holmboe
JAMA. 2006;296(9):1137-1139.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Why Medical Educators May Be Failing at Feedback
Bing-You and Trowbridge
JAMA 2009;302:1330-1331.
FULL TEXT  

Why Don't We Talk?
Jones
Arch Pediatr Adolesc Med 2009;163:865-866.
FULL TEXT  

A Multisource and Repeated Measure Approach to Assessing Patient--Physician Relationship and Patient Satisfaction
Weng
Eval Health Prof 2009;32:128-143.
ABSTRACT  

Medical Education Theme Issue 2009: Call for Papers
Golub
JAMA 2009;301:972-972.
FULL TEXT  

Revalidation of Canadian physicians
Levinson
CMAJ 2009;180:540-541.
FULL TEXT  

Do Self-Assessments Work to Detect Workshop Success?: An Analysis of Argument and Recommendation by D'Eon et al
Lam
American Journal of Evaluation 2009;30:93-105.
ABSTRACT  

The Science of Continuing Medical Education: Terms, Tools, and Gaps: Effectiveness of Continuing Medical Education: American College of Chest Physicians Evidence-Based Educational Guidelines
Davis et al.
Chest 2009;135:8S-16S.
ABSTRACT | FULL TEXT  

Education Techniques for Lifelong Learning: Lifelong Learning in the 21st Century and Beyond1
Collins
RadioGraphics 2009;29:613-622.
ABSTRACT | FULL TEXT  

Identifying Areas for Curricular Program Improvement Based on Perceptions of Skills, Competencies, and Performance
Gathright et al.
Acad. Psychiatry 2009;33:37-42.
ABSTRACT | FULL TEXT  

CPD and recertification: improving patient outcomes through focused learning
Bamrah and Bhugra
Adv. Psychiatr. Treat. 2009;15:2-6.
ABSTRACT | FULL TEXT  

Pediatric Fellows: Perspectives on Training and Future Scope of Practice
Freed et al.
Pediatrics 2009;123:S31-S37.
ABSTRACT | FULL TEXT  

Recently Trained Pediatric Subspecialists: Perspectives on Training and Scope of Practice
Freed et al.
Pediatrics 2009;123:S44-S49.
ABSTRACT | FULL TEXT  

What Can Data Tell Us About the Quality and Relevance of Current Pediatric Residency Education?
Leslie
Pediatrics 2009;123:S50-S55.
ABSTRACT | FULL TEXT  

Pediatric Resident Confidence in Resuscitation Skills Relates to Mock Code Experience
van Schaik et al.
CLIN PEDIATR 2008;47:777-783.
ABSTRACT  

Continuing medical education in the 21st century
Woollard
BMJ 2008;337:a119-a119.
FULL TEXT  

Invited Article: The ABPN maintenance of certification program for neurologists: Past, present, and future
Faulkner et al.
Neurology 2008;71:599-604.
FULL TEXT  

The effectiveness of an educational programme on occupational disease reporting
Smits et al.
Occup Med (Lond) 2008;58:373-375.
ABSTRACT | FULL TEXT  

Questionnaires for 360-degree assessment of consultant psychiatrists: development and psychometric properties
Lelliott et al.
Br. J. Psychiatry 2008;193:156-160.
ABSTRACT | FULL TEXT  

Problem based learning
Wood
BMJ 2008;336:971-971.
FULL TEXT  

The ABPN Maintenance of Certification Program for Psychiatrists: Past History, Current Status, and Future Directions
Faulkner et al.
Acad. Psychiatry 2008;32:241-248.
ABSTRACT | FULL TEXT  

The self critical doctor: helping students become more reflective
Driessen et al.
BMJ 2008;336:827-830.
FULL TEXT  

Lifelong learning at work
Teunissen and Dornan
BMJ 2008;336:667-669.
FULL TEXT  

Invited article: Neurology education research
Stern et al.
Neurology 2008;70:876-883.
ABSTRACT | FULL TEXT  

Dental Students' Self-Assessment of Preclinical Examinations
Curtis et al.
J Dent Educ 2008;72:265-277.
ABSTRACT | FULL TEXT  

Using Self-Assessments to Detect Workshop Success: Do They Work?
D'Eon et al.
American Journal of Evaluation 2008;29:92-98.
ABSTRACT  

Impact of Research-based Synopses Delivered as Daily E-mail: A Prospective Observational Study
Grad et al.
J. Am. Med. Inform. Assoc. 2008;15:240-245.
ABSTRACT | FULL TEXT  

Teaching when time is limited
Irby and Wilkerson
BMJ 2008;336:384-387.
FULL TEXT  

Health staff perception regarding quality of delivered information to inpatients
Duclos et al.
Int J Qual Health Care 2008;20:13-21.
ABSTRACT | FULL TEXT  

The effects of problem-based learning during medical school on physician competency: a systematic review
Koh et al.
CMAJ 2008;178:34-41.
ABSTRACT | FULL TEXT  

Improving Asthma Care Through Recertification: A Cluster Randomized Trial
Simpkins et al.
Arch Intern Med 2007;167:2240-2248.
ABSTRACT | FULL TEXT  

Effectiveness of Teaching Quality Improvement to Clinicians: A Systematic Review
Boonyasai et al.
JAMA 2007;298:1023-1037.
ABSTRACT | FULL TEXT  

Translating learning into practice: Lessons from the practice-based small group learning program
Armson et al.
cfp 2007;53:1477-1485.
ABSTRACT | FULL TEXT  

Efficacy of Communication Skills Training for Giving Bad News and Discussing Transitions to Palliative Care
Back et al.
Arch Intern Med 2007;167:453-460.
ABSTRACT | FULL TEXT  

Sophia
Emerg. Med. J. 2007;24:70-70.
FULL TEXT  

Self-assessment in lifelong learning and improving performance in practice: physician know thyself.
Duffy and Holmboe
JAMA 2006;296:1137-1139.
FULL TEXT  

Medical education 2006: beyond mental mediocrity.
Golub
JAMA 2006;296:1139-1140.
FULL TEXT