This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (50)  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Topic Collections  • Informatics/ Internet in Medicine  • Internet  • Quality of Care, Other  • Alert me on articles by topic

The Risk of Quality Ratings Based on Mortality Data

Harlan M. Krumholz, MD; Saif S. Rathore, MPH; Jersey Chen, MD, MPH; Yongfei Wang, MS; Martha J. Radford, MD

JAMA. 2002;287:1277-1287.

Context  Health care "report cards" have attracted significant consumer interest, particularly publicly available Internet health care quality rating systems. However, the ability of these ratings to discriminate between hospitals is not known.

Objective  To determine whether hospital ratings for acute myocardial infarction (AMI) mortality from a prominent Internet hospital rating system accurately discriminate between hospitals' performance based on process of care and outcomes.

Design, Setting, and Patients  Data from the Cooperative Cardiovascular Project, a retrospective systematic medical record review of 141 914 Medicare fee-for-service beneficiaries 65 years or older hospitalized with AMI at 3363 US acute care hospitals during a 4- to 8-month period between January 1994 and February 1996 were compared with ratings obtained from HealthGrades.com (1-star: worse outcomes than predicted, 5-star: better outcomes than predicted) based on 1994-1997 Medicare data.

Main Outcome Measures  Quality indicators of AMI care, including use of acute reperfusion therapy, aspirin, -blockers, angiotensin-converting enzyme inhibitors; 30-day mortality.

Results  Patients treated at higher-rated hospitals were significantly more likely to receive aspirin (admission: 75.4% 5-star vs 66.4% 1-star, P for trend = .001; discharge: 79.7% 5-star vs 68.0% 1-star, P = .001) and -blockers (admission: 54.8% 5-star vs 35.7% 1-star, P = .001; discharge: 63.3% 5-star vs 52.1% 1-star, P = .001), but not angiotensin-converting enzyme inhibitors (59.6% 5-star vs 57.4% 1-star, P = .40). Acute reperfusion therapy rates were highest for patients treated at 2-star hospitals (60.6%) and lowest for 5-star hospitals (53.6% 5-star, P = .008). Risk-standardized 30-day mortality rates were lower for patients treated at higher-rated than lower-rated hospitals (21.9% 1-star vs 15.9% 5-star, P = .001). However, there was marked heterogeneity within rating groups and substantial overlap of individual hospitals across rating strata for mortality and process of care; only 3.1% of comparisons between 1-star and 5-star hospitals had statistically lower risk-standardized 30-day mortality rates in 5-star hospitals. Similar findings were observed in comparisons of 30-day mortality rates between individual hospitals in all other rating groups and when comparisons were restricted to hospitals with a minimum of 30 cases during the study period.

Conclusion  Hospital ratings published by a prominent Internet health care quality rating system identified groups of hospitals that, in the aggregate, differed in their quality of care and outcomes. However, the ratings poorly discriminated between any 2 individual hospitals' process of care or mortality rates during the study period. Limitations in discrimination may undermine the value of health care quality ratings for patients or payers and may lead to misperceptions of hospitals' performance.

Author Affiliations: Section of Cardiovascular Medicine, Department of Medicine (Drs Krumholz, Chen, and Radford, and Messrs Rathore and Wang), and Section of Health Policy and Administration, Department of Epidemiology and Public Health (Dr Krumholz), Yale University School of Medicine, New Haven, Conn; Yale-New Haven Hospital Center for Outcomes Research and Evaluation, New Haven, Conn (Drs Krumholz and Radford); and Qualidigm, Middletown, Conn (Drs Krumholz and Radford). Dr Chen is currently affiliated with the Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia.

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Hospital mortality: when failure is not a good measure of success
Shojania and Forster
CMAJ 2008;179:153-157.
FULL TEXT  

User Acceptance of HIV TIDES--Tailored Interventions for Management of Depressive Symptoms in Persons Living with HIV/AIDS
Lai et al.
J. Am. Med. Inform. Assoc. 2008;15:217-226.
ABSTRACT | FULL TEXT  

Publicly Available Hospital Comparison Web Sites: Determination of Useful, Valid, and Appropriate Information for Comparing Surgical Quality
Leonardi et al.
Arch Surg 2007;142:863-869.
ABSTRACT | FULL TEXT  

REPRINT Treatment of Hypertension in the Prevention and Management of Ischemic Heart Disease: A Scientific Statement From the American Heart Association Council for High Blood Pressure Research and the Councils on Clinical Cardiology and Epidemiology and Prevention
Rosendorff et al.
Hypertension 2007;50:e28-e55.
FULL TEXT  

Identifying High-Quality Hospitals: Consult the Ratings or Flip a Coin?
O'Brien and Peterson
Arch Intern Med 2007;167:1342-1344.
FULL TEXT  

"America's Best Hospitals" in the Treatment of Acute Myocardial Infarction
Wang et al.
Arch Intern Med 2007;167:1345-1351.
ABSTRACT | FULL TEXT  

Treatment of Hypertension in the Prevention and Management of Ischemic Heart Disease: A Scientific Statement From the American Heart Association Council for High Blood Pressure Research and the Councils on Clinical Cardiology and Epidemiology and Prevention
Rosendorff et al.
Circulation 2007;115:2761-2788.
FULL TEXT  

Comparison of Clinical and Administrative Data Sources for Hospital Coronary Artery Bypass Graft Surgery Report Cards
Shahian et al.
Circulation 2007;115:1518-1527.
ABSTRACT | FULL TEXT  

Pay-for-Performance: Will the Latest Payment Trend Improve Care?
Rosenthal and Dudley
JAMA 2007;297:740-744.
FULL TEXT  

Public Reporting and Pay for Performance in Hospital Quality Improvement
Lindenauer et al.
NEJM 2007;356:486-496.
ABSTRACT | FULL TEXT  

Performance of Top-Ranked Heart Care Hospitals on Evidence-Based Process Measures
Williams et al.
Circulation 2006;114:558-564.
ABSTRACT | FULL TEXT  

Will physician-level measures of clinical performance be used in medical malpractice litigation?
Kesselheim et al.
JAMA 2006;295:1831-1834.
FULL TEXT  

Development and Validation of a Prognostic Index for 4-Year Mortality in Older Adults
Lee et al.
JAMA 2006;295:801-808.
ABSTRACT | FULL TEXT  

Public Reporting and Case Selection for Percutaneous Coronary Interventions: An Analysis From Two Large Multicenter Percutaneous Coronary Intervention Databases
Moscucci et al.
J Am Coll Cardiol 2005;45:1759-1765.
ABSTRACT | FULL TEXT  

Quality and Quality Improvement in Forensic Mental Health Evaluations
Wettstein
J Am Acad Psychiatry Law 2005;33:158-175.
ABSTRACT | FULL TEXT  

Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes?
Mack et al.
J. Thorac. Cardiovasc. Surg. 2005;129:1309-1317.
ABSTRACT | FULL TEXT  

A Middle Ground on Public Accountability
Lee et al.
NEJM 2004;350:2409-2412.
FULL TEXT  

Patients' Interest in Reading Their Medical Record: Relation With Clinical and Sociodemographic Characteristics and Patients' Approach to Health Care
Fowles et al.
Arch Intern Med 2004;164:793-800.
ABSTRACT | FULL TEXT  

Physician Clinical Performance Assessment: Prospects and Barriers
Landon et al.
JAMA 2003;290:1183-1189.
ABSTRACT | FULL TEXT  

Quality Report Cards and Nursing Home Quality
Mukamel and Spector
Gerontologist 2003;43:58-66.
ABSTRACT | FULL TEXT  

Employers' Efforts To Measure And Improve Hospital Quality: Determinants Of Success
Mehrotra et al.
Health Aff (Millwood) 2003;22:60-71.
ABSTRACT | FULL TEXT  

Large Employers' New Strategies in Health Care
Galvin and Milstein
NEJM 2002;347:939-942.
FULL TEXT  

Considerations in the Development of Intensive Care Unit Report Cards
Gupta et al.
J Intensive Care Med 2002;17:211-217.
ABSTRACT  

Quality of Care in Patients with Diabetes
Greenfield et al.
ANN INTERN MED 2002;137:71-72.
FULL TEXT  

Variations in mortality rates among Canadian NICUs --and anonymous reporting
Lee et al.
CMAJ 2002;167:120-121.
FULL TEXT  

Should Consumers Trust Hospital Quality Report Cards?
Finlayson et al.
JAMA 2002;287:3206-3208.
FULL TEXT  

Public Profiling of Clinical Performance
Naylor
JAMA 2002;287:1323-1325.
FULL TEXT