This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (52)  • Contact me when this article is cited  Related Content  • Related letters  • Similar articles in JAMA  Topic Collections  • Oncology  • Colon Cancer  • Quality of Care, Other  • Diagnosis  • Alert me on articles by topic

Lessons Learned From a VA Performance Measure

Louise C. Walter, MD; Natalie P. Davidowitz, BA; Paul A. Heineken, MD; Kenneth E. Covinsky, MD, MPH

JAMA. 2004;291:2466-2470.

The Department of Veterans Affairs (VA) manages the largest health care system in the United States, and the Institute of Medicine has recommended that many practices of VA quality measurement be applied to the US health care system as a whole. The VA measures quality of care at all of its sites by assessing adherence rates to performance measures, which generally are derived from evidence-based practice guidelines. Higher adherence rates are used as evidence of better quality of care. However, there are problems with converting practice guidelines, intended to offer guidance to clinicians, into performance measures that are meant to identify poor-quality care. We suggest a more balanced perspective on the use of performance measures to define quality by delineating conceptual problems with the conversion of practice guidelines into quality measures. Focusing on colorectal cancer screening, we use a case study at 1 VA facility to illustrate pitfalls that can cause adherence rates to guideline-based performance measures to be poor indicators of the quality of cancer screening. Pitfalls identified included (1) not properly considering illness severity of the sample population audited for adherence to screening, (2) not distinguishing screening from diagnostic procedures when setting achievable target screening rates, and (3) not accounting for patient preferences or clinician judgment when scoring performance measures. For many patients with severe comorbid illnesses or strong preferences against screening, the risks of colorectal cancer screening outweigh the benefits, and the decision to not screen may reflect good quality of care. Performance measures require more thoughtful specification and interpretation to avoid defining high testing rates as good quality of care regardless of who received the test, why it was performed, or whether the patient wanted it.

Author Affiliations: Division of Geriatrics, San Francisco VA Medical Center, and the University of California, San Francisco (Drs Walter and Covinsky and Ms Davidowitz); and San Francisco VA Medical Center (Dr Heineken).

RELATED LETTERS

Pitfalls of Converting Practice Guidelines Into Quality Measures
Greg Pawlson, Sarah Hudson Scholle, and Philip Renner
JAMA. 2004;292(11):1301.
EXTRACT | FULL TEXT  

Pitfalls of Converting Practice Guidelines Into Quality Measures
Barbara B. Fleming and Robert A. Petzel
JAMA. 2004;292(11):1301-1302.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Should we use large scale healthcare interventions without clear evidence that benefits outweigh costs and harms? No
Landefeld et al.
BMJ 2008;336:1277-1277.
FULL TEXT  

Assessment of the Scientific Soundness of Clinical Performance Measures: A Field Test of the National Committee for Quality Assurance's Colorectal Cancer Screening Measure
Schneider et al.
Arch Intern Med 2008;168:876-882.
ABSTRACT | FULL TEXT  

Improving the Quality of Measurement and Evaluation in Quality Improvement Efforts
Pronovost et al.
American Journal of Medical Quality 2008;23:143-146.
 

The Quality Oncology Practice Initiative (QOPI): An Important Tool for Measuring Quality and Improving Care
Jacobson
Am Soc Clin Oncol Ed Book 2008;2008:501-505.
ABSTRACT | FULL TEXT  

Do we know quality epilepsy care when we see it?
Langfitt
Neurology 2007;69:1968-1969.
FULL TEXT  

Maximizing Informed Cancer Screening Decisions
Walter and Lewis
Arch Intern Med 2007;167:2027-2028.
FULL TEXT  

Social and Racial Differences in Selection of Breast Cancer Adjuvant Chemotherapy Regimens
Griggs et al.
JCO 2007;25:2522-2527.
ABSTRACT | FULL TEXT  

Relationship between Number of Medical Conditions and Quality of Care
Higashi et al.
NEJM 2007;356:2496-2504.
ABSTRACT | FULL TEXT  

Clinical Concerns About Clinical Performance Measurement
Werner and Asch
Ann Fam Med 2007;5:159-163.
ABSTRACT | FULL TEXT  

Automated Review of Electronic Health Records to Assess Quality of Care for Outpatients with Heart Failure
Baker et al.
ANN INTERN MED 2007;146:270-277.
ABSTRACT | FULL TEXT  

Public Reporting and Pay for Performance in Hospital Quality Improvement
Lindenauer et al.
NEJM 2007;356:486-496.
ABSTRACT | FULL TEXT  

American Psychiatric Association Practice Guidelines for the Treatment of Psychiatric Disorders Compendium 2006
Glass
JAMA 2006;296:2620-2621.
FULL TEXT  

PSA Screening Among Elderly Men With Limited Life Expectancies.
Walter et al.
JAMA 2006;296:2336-2342.
ABSTRACT | FULL TEXT  

Colorectal cancer screening in young patients with poor health and severe comorbidity.
Sultan et al.
Arch Intern Med 2006;166:2209-2214.
ABSTRACT | FULL TEXT  

Assessing the validity of national quality measures for coronary artery disease using an electronic health record.
Persell et al.
Arch Intern Med 2006;166:2272-2277.
ABSTRACT | FULL TEXT  

The effect of age and chronic illness on life expectancy after a diagnosis of colorectal cancer: implications for screening.
Gross et al.
ANN INTERN MED 2006;145:646-653.
ABSTRACT | FULL TEXT  

Reconsidering medication appropriateness for patients late in life.
Holmes et al.
Arch Intern Med 2006;166:605-609.
FULL TEXT  

Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology.
Green and Glasgow
Eval Health Prof 2006;29:126-153.
ABSTRACT  

A New ATS Committee: Competing in the Marketplace of Ideas
Schunemann and Heffner
Am. J. Respir. Cell Mol. Bio. 2005;33:423-424.
FULL TEXT  

A New ATS Committee: Competing in the Marketplace of Ideas
Schunemann and Heffner
Proc Am Thorac Soc 2005;2:249-250.
FULL TEXT  

A New ATS Committee: Competing in the Marketplace of Ideas
Schunemann and Heffner
Am. J. Respir. Crit. Care Med. 2005;172:1067-1068.
FULL TEXT  

An Automated Data Algorithm To Distinguish Screening and Diagnostic Colorectal Cancer Endoscopy Exams
Haque et al.
J Natl Cancer Inst Monogr 2005;2005:116-118.
ABSTRACT | FULL TEXT  

Moving beyond Round Pegs and Square Holes: Restructuring Medicare To Improve Chronic Care
Wolff and Boult
ANN INTERN MED 2005;143:439-445.
ABSTRACT | FULL TEXT  

Clinical Practice Guidelines and Quality of Care for Older Patients With Multiple Comorbid Diseases: Implications for Pay for Performance
Boyd et al.
JAMA 2005;294:716-724.
ABSTRACT | FULL TEXT  

Screening for Alcohol Use Disorders Among Medical Outpatients: The Influence of Individual and Facility Characteristics
Desai et al.
Am. J. Psychiatry 2005;162:1521-1526.
ABSTRACT | FULL TEXT  

Quality and Quality Improvement in Forensic Mental Health Evaluations
Wettstein
J Am Acad Psychiatry Law 2005;33:158-175.
ABSTRACT | FULL TEXT  

The Unintended Consequences of Publicly Reporting Quality Information
Werner and Asch
JAMA 2005;293:1239-1244.
ABSTRACT | FULL TEXT  

Today's FDA
Slater
NEJM 2005;352:293-297.
FULL TEXT  

Evidence Based? Caveat Emptor!
Steinberg and Luce
Health Aff (Millwood) 2005;24:80-92.
ABSTRACT | FULL TEXT  

Evidence-Based Guidelines As a Foundation For Performance Incentives
Garber
Health Aff (Millwood) 2005;24:174-179.
ABSTRACT | FULL TEXT  

Payment for Quality in End-Stage Renal Disease
Himmelfarb et al.
J. Am. Soc. Nephrol. 2004;15:3263-3269.
FULL TEXT  

Pitfalls of Converting Practice Guidelines Into Quality Measures
Pawlson et al.
JAMA 2004;292:1301-1301.
FULL TEXT  

Pitfalls of Converting Practice Guidelines Into Quality Measures
Fleming and Petzel
JAMA 2004;292:1301-1302.
FULL TEXT