This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (109)  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

When Good Apples Look Bad

Susanne Salem-Schatz, ScD; Gordon Moore, MD, MPH; Malcolm Rucker, MS; Steven D. Pearson, MD, MS

JAMA. 1994;272(11):871-874.

Abstract
Objective.
—To assess the influence of patient characteristics on practice profiling. Using the example of specialty referrals by primary care physicians, we evaluated the impact of adjusting for patient characteristics (age/sex vs case mix) on the estimation of practice variation, the identification of outlier practices, and the evaluation of potential predictors of variation.

Design and Setting.
—We applied several measurement strategies to a retrospective cohort of patients (N=37 830) within 52 physician practices in a large staffmodel health maintenance organization during a 1-year period.

Outcome Measures.
—We calculated unadjusted referral rates and adjusted standardized referral ratios for each physician. Using these, we determined coefficients of variation and statistical "outlier status."

Results.
—Adjustment for patient characteristics decreased the observed variation in referral profiles, with a decrease of more than 50% in the coefficient of variation. Three quarters of the physicians identified as statistical outliers with use of an age/sex—adjusted measure were no longer identified as such with use of a case-mix—adjusted measure. Several key predictors of unadjusted referral rate (including physician age, practice tenure, site of practice, and extent of laboratory test ordering) dropped out of regression models when the outcome variable was adjusted for patient characteristics.

Conclusion.
—Failure to adjust for case mix in physician practice profiles may lead to overestimates of variation and misidentification of outliers. To the extent that unadjusted practice profiles are used for decisions about education, sanctions, or employment, physicians may be subject to inequitable decisions and actions. Misinformation about the causes and extent of practice variation may also lead to misdirection of scarce resources for quality improvement efforts.

(JAMA. 1994;272:871-874)

Author Affiliations
From the Harvard Community Health Plan, Brookline, Mass.

Footnotes
Reprint requests to Harvard Community Health Plan, Clinical Quality Management, 10 Brookline PI W, 2nd Floor, Brookline, MA 02146 (Dr Salem-Schatz).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study
Forrest et al.
Ann Fam Med 2007;5:361-367.
ABSTRACT | FULL TEXT  

Primary Care Physician Specialty Referral Decision Making: Patient, Physician, and Health Care System Determinants
Forrest et al.
Med Decis Making 2006;26:76-85.
ABSTRACT  

Effects of Comorbidity and Clustering upon Referrals in Primary Care
Chen et al.
J Am Board Fam Med 2005;18:449-452.
ABSTRACT | FULL TEXT  

Why is the grass greener?
Starfield
BMJ 2005;330:727-729.
FULL TEXT  

Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example
Huang et al.
Med Decis Making 2005;25:20-34.
ABSTRACT  

Adjusting for case mix and social class in examining variation in home visits between practices
Sullivan et al.
Fam Pract 2004;21:355-363.
ABSTRACT | FULL TEXT  

Risk Adjustment for Pediatric Quality Indicators
Kuhlthau et al.
Pediatrics 2004;113:210-216.
ABSTRACT | FULL TEXT  

On Being New to an Insurance Plan: Health Care Use Associated With the First Years in a Health Insurance Plan
Franks et al.
Ann Fam Med 2003;1:156-161.
ABSTRACT | FULL TEXT  

The impact of statistical adjustment on economic profiles of interventional cardiologists
Cowper et al.
J Am Coll Cardiol 2001;38:1416-1423.
ABSTRACT | FULL TEXT  

Physician profiling in the catheterization laboratory: a worthwhile strategy or a path to futility?
Vaitkus
J Am Coll Cardiol 2001;38:1424-1426.
FULL TEXT  

Use of risk adjustment in setting budgets and measuring performance in primary care II: advantages, disadvantages, and practicalities
Majeed et al.
BMJ 2001;323:607-610.
FULL TEXT  

Challenges in comparing risk-adjusted bypass surgery mortality results: Results from the Cooperative Cardiovascular Project
Peterson et al.
J Am Coll Cardiol 2000;36:2174-2184.
ABSTRACT | FULL TEXT  

Comparison of Vignettes, Standardized Patients, and Chart Abstraction: A Prospective Validation Study of 3 Methods for Measuring Quality
Peabody et al.
JAMA 2000;283:1715-1722.
ABSTRACT | FULL TEXT  

Influence of Patient Education on Profiles of Physician Practices
Fiscella and Franks
ANN INTERN MED 1999;131:745-751.
ABSTRACT | FULL TEXT  

Pediatric Physician Profiling
Committee on Practice and Ambulatory Medicine and
Pediatrics 1999;104:970-972.
ABSTRACT | FULL TEXT  

The Pediatric Primary-Specialty Care Interface: How Pediatricians Refer Children and Adolescents to Specialty Care
Forrest et al.
Arch Pediatr Adolesc Med 1999;153:705-714.
ABSTRACT | FULL TEXT  

Use of Standardized Patients to Assess Between-Physician Variations in Resource Utilization
McLeod et al.
JAMA 1997;278:1164-1168.
ABSTRACT  

Case-Mix Adjustment Using Administrative Databases: A Paradigm to Guide Future Research
Wray et al.
Med Care Res Rev 1997;54:326-356.
ABSTRACT  

Preserving Community in Health Care
Emanuel and Emanuel
Journal of Health Politics, Policy and Law 1997;22:147-184.
ABSTRACT  

Taking Health Status Into Account When Setting Capitation Rates: A Comparison of Risk-Adjustment Methods
Fowles et al.
JAMA 1996;276:1316-1321.
ABSTRACT  

Case-Mix Adjustment: Making Bad Apples Look Good
Welch et al.
JAMA 1995;273:772-773.
ABSTRACT  

Preserving the Physician-Patient Relationship in the Era of Managed Care
Emanuel and Dubler
JAMA 1995;273:323-329.
ABSTRACT