This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (263)  • Contact me when this article is cited  Related Content  • Related letter  • Related articles  • Similar articles in JAMA  Topic Collections  • Medical Practice  • Medical Practice, Other  • Quality of Care  • Quality of Care, Other  • Alert me on articles by topic  Social Bookmarking   What's this?

What Do We Expect to Gain? A Review of the Evidence

Martin N. Marshall, MSc, MD, FRCGP; Paul G. Shekelle, MD, PhD; Sheila Leatherman, MSW; Robert H. Brook, MD, ScD

JAMA. 2000;283:1866-1874.

Context  Information about the performance of hospitals, health professionals, and health care organizations has been made public in the United States for more than a decade. The expected gains of public disclosure have not been made clear, and both the benefits and potential risks have received minimal empirical investigation.

Objective  To summarize the empirical evidence concerning public disclosure of performance data, relate the results to the potential gains, and identify areas requiring further research.

Data Sources  A literature search was conducted on MEDLINE and EMBASE databases for articles published between January 1986 and October 1999 in peer-reviewed journals. Review of citations, public documents, and expert advice was conducted to identify studies not found in the electronic databases.

Study Selection  Descriptive, observational, or experimental evaluations of US reporting systems were selected for inclusion.

Data Extraction  Included studies were organized based on use of public data by consumers, purchasers, physicians, and hospitals; impact on quality of care outcomes; and costs.

Data Synthesis  Seven US reporting systems have been the subject of published empirical evaluations. Descriptive and observational methods predominate. Consumers and purchasers rarely search out the information and do not understand or trust it; it has a small, although increasing, impact on their decision making. Physicians are skeptical about such data and only a small proportion makes use of it. Hospitals appear to be most responsive to the data. In a limited number of studies, the publication of performance data has been associated with an improvement in health outcomes.

Conclusions  There are several potential gains from the public disclosure of performance data, but use of the information by provider organizations for quality improvement may be the most productive area for further research.

Author Affiliations: School of Postgraduate Medicine and Health Sciences, University of Exeter, Exeter, England (Dr Marshall); RAND Health Program, Santa Monica (Drs Shekelle and Brook), Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles (Dr Shekelle), Calif; Center for Health Care Policy and Evaluation, United Health Group, Minnetonka, Minn (Ms Leatherman); Judge Institute of Management, University of Cambridge, Cambridge, England (Ms Leatherman); and University of California, Los Angeles, Center for Health Services, Los Angeles (Dr Brook). Dr Marshall is now with the National Primary Care Research and Development Centre, University of Manchester, Manchester, England.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati     What's this?

RELATED LETTER

Use of Performance Data to Change Physician Behavior
Alan H. Rosenstein, Martin N. Marshall, Paul G. Shekelle, Robert H. Brook, and Sheila Leatherman
JAMA. 2000;284(9):1079.
EXTRACT | FULL TEXT  

RELATED ARTICLES

Public Release of Performance Data: A Progress Report From the Front
Arnold M. Epstein
JAMA. 2000;283(14):1884-1886.
EXTRACT | FULL TEXT  

April 12, 2000
JAMA. 2000;283(14):1905-1906.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Is There Evidence of Cream Skimming Among Nursing Homes Following the Publication of the Nursing Home Compare Report Card?
Mukamel et al.
The Gerontologist 2009;0:gnp062v1-gnp062.
ABSTRACT | FULL TEXT  

2009 Focused Update Incorporated Into the ACC/AHA 2005 Guidelines for the Diagnosis and Management of Heart Failure in Adults: A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines Developed in Collaboration With the International Society for Heart and Lung Transplantation
Hunt et al.
J Am Coll Cardiol 2009;53:e1-e90.
FULL TEXT  

2009 Focused Update Incorporated Into the ACC/AHA 2005 Guidelines for the Diagnosis and Management of Heart Failure in Adults: A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines: Developed in Collaboration With the International Society for Heart and Lung Transplantation
2005 WRITING COMMITTEE MEMBERS et al.
Circulation 2009;119:e391-e479.
FULL TEXT  

Communicating Hospital Infection Data to the Public: A Study of Consumer Responses and Preferences
Mazor et al.
American Journal of Medical Quality 2009;24:108-115.
ABSTRACT  

Motivating Public Use of Physician-Level Performance Data: An Experiment on the Effects of Message and Mode
Ranganathan et al.
Med Care Res Rev 2009;66:68-81.
ABSTRACT  

Contextualization of Physical and Sexual Assault in Male Prisons: Incidents and Their Aftermath
Wolff and Jing Shi
J Correct Health Care 2009;15:58-77.
ABSTRACT  

Public Reporting of Quality Data for Stroke: Is It Measuring Quality?
Kelly et al.
Stroke 2008;39:3367-3371.
ABSTRACT | FULL TEXT  

Nonprimary Care Physicians' Views on Office-Based Quality Incentive and Improvement Programs
Murphy and Nash
American Journal of Medical Quality 2008;23:427-439.
ABSTRACT  

Regulation, measurements and incentives. The experience in the US and UK: does context matter?
Hamblin
The Journal of the Royal Society for the Promotion of Health 2008;128:291-298.
ABSTRACT  

Choosing The Best Hospital: The Limitations Of Public Quality Reporting
Rothberg et al.
Health Aff (Millwood) 2008;27:1680-1687.
ABSTRACT | FULL TEXT  

Willingness of patients to change surgeons for a shorter waiting time for joint arthroplasty
Conner-Spady et al.
CMAJ 2008;179:327-332.
ABSTRACT | FULL TEXT  

Is choice working for patients in the English NHS?
Bevan
BMJ 2008;337:a935-a935.
FULL TEXT  

What benefits will choice bring to patients? Literature review and assessment of implications
Fotaki et al.
J Health Serv Res Policy 2008;13:178-184.
ABSTRACT | FULL TEXT  

ACCF 2008 Health Policy Statement on Principles for Public Reporting of Physician Performance Data: A Report of the American College of Cardiology Foundation Writing Committee to Develop Principles for Public Reporting of Physician Performance Data
Drozda et al.
J Am Coll Cardiol 2008;51:1993-2001.
FULL TEXT  

Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care
Fung et al.
ANN INTERN MED 2008;148:111-123.
ABSTRACT | FULL TEXT  

Defining poor and optimum performance in an IVF programme
Castilla et al.
Hum Reprod 2008;23:85-90.
ABSTRACT | FULL TEXT  

Risk Stratification and Comorbidity
Ferraris et al.
Card Surg Adult 2008;3:199-246.
FULL TEXT  

Predictors of Timely Follow-Up Care Among Medicaid-Enrolled Adults After Psychiatric Hospitalization
Stein et al.
Psychiatr. Serv. 2007;58:1563-1569.
ABSTRACT | FULL TEXT  

Promoting quality: the health-care organization from a management perspective
Glickman et al.
Int J Qual Health Care 2007;19:341-348.
ABSTRACT | FULL TEXT  

Do we know quality epilepsy care when we see it?
Langfitt
Neurology 2007;69:1968-1969.
FULL TEXT  

Improving the quality of care with performance indicators
Majeed et al.
BMJ 2007;335:916-918.
FULL TEXT  

Publicly Available Hospital Comparison Web Sites: Determination of Useful, Valid, and Appropriate Information for Comparing Surgical Quality
Leonardi et al.
Arch Surg 2007;142:863-869.
ABSTRACT | FULL TEXT  

Publishing 'quality' measures: how it works and when it does not?
Hamblin
Int J Qual Health Care 2007;19:183-186.
FULL TEXT  

Relationship between Number of Medical Conditions and Quality of Care
Higashi et al.
NEJM 2007;356:2496-2504.
ABSTRACT | FULL TEXT  

A comparison of quality of care indicators in urban acute care hospitals and rural critical access hospitals in the United States
Lutfiyya et al.
Int J Qual Health Care 2007;19:141-149.
ABSTRACT | FULL TEXT  

Choosing Where to Have Major Surgery: Who Makes the Decision?
Wilson et al.
Arch Surg 2007;142:242-246.
ABSTRACT | FULL TEXT  

Public Reporting and Pay for Performance in Hospital Quality Improvement
Lindenauer et al.
NEJM 2007;356:486-496.
ABSTRACT | FULL TEXT  

Measuring Performance For Treating Heart Attacks And Heart Failure: The Case For Outcomes Measurement
Krumholz et al.
Health Aff (Millwood) 2007;26:75-85.
ABSTRACT | FULL TEXT  

Relationship Between Medicare's Hospital Compare Performance Measures and Mortality Rates
Werner and Bradlow
JAMA 2006;296:2694-2702.
ABSTRACT | FULL TEXT  

The Impact of Unmeasured Clinical Variables on the Accuracy of Hospital Report Cards: A Monte Carlo Study
Austin
Med Decis Making 2006;26:447-466.
ABSTRACT  

The Impact Of Quality-Reporting Programs On Hospital Operations
Pham et al.
Health Aff (Millwood) 2006;25:1412-1422.
ABSTRACT | FULL TEXT  

Do Cardiac Surgery Report Cards Reduce Mortality? Assessing the Evidence
Epstein
Med Care Res Rev 2006;63:403-426.
ABSTRACT  

Is a public reporting approach appropriate for nursing home care?
Stevenson
Journal of Health Politics, Policy and Law 2006;31:773-810.
ABSTRACT  

Paying for performance in the United States and abroad.
Epstein
NEJM 2006;355:406-408.
FULL TEXT  

Strategies to Enhance Price and Quality Competition in Health Care: Lessons Learned from Tracking Local Markets
Lesser and Ginsburg
Journal of Health Politics, Policy and Law 2006;31:557-567.
ABSTRACT  

Strategies for improving surgical quality--should payers reward excellence or effort?
Birkmeyer and Birkmeyer
NEJM 2006;354:864-870.
FULL TEXT  

Reframing the Racial Disparities Issue for State Governments
Stone
Journal of Health Politics, Policy and Law 2006;31:127-152.
ABSTRACT  

Using markets to reform health care
Edwards
BMJ 2005;331:1464-1466.
FULL TEXT  

Regional Variation in Rates of Low Birth Weight
Thompson et al.
Pediatrics 2005;116:1114-1121.
ABSTRACT | FULL TEXT  

How do elderly patients decide where to go for major surgery? Telephone interview survey
Schwartz et al.
BMJ 2005;331:821.
ABSTRACT | FULL TEXT  

Quality of Care in U.S. Hospitals as Reflected by Standardized Measures, 2002-2004
Williams et al.
NEJM 2005;353:255-264.
ABSTRACT | FULL TEXT  

Administrative Data Feedback for Effective Cardiac Treatment: AFFECT, A Cluster Randomized Trial
Beck et al.
JAMA 2005;294:309-317.
ABSTRACT | FULL TEXT  

The Cost-Quality Trade-Off: Need for Data Quality Standards for Studies That Impact Clinical Practice and Health Policy
Malin and Keating
JCO 2005;23:4581-4584.
FULL TEXT  

Hospital Performance Reports: Impact On Quality, Market Share, And Reputation
Hibbard et al.
Health Aff (Millwood) 2005;24:1150-1160.
ABSTRACT | FULL TEXT  

It Isn't Just about Choice: The Potential of a Public Performance Report to Affect the Public Image of Hospitals
Hibbard et al.
Med Care Res Rev 2005;62:358-371.
ABSTRACT  

Improving the ICU: Part 2
Garland
Chest 2005;127:2165-2179.
ABSTRACT | FULL TEXT  

Fostering Rational Regulation of Patient Safety
Mello et al.
Journal of Health Politics, Policy and Law 2005;30:375-426.
ABSTRACT  

Nursing Home Administrators' Opinions of the Nursing Home Compare Web Site
Castle
Gerontologist 2005;45:299-308.
ABSTRACT | FULL TEXT  

Impact of reporting hospital performance
Marshall and Romano
Qual Saf Health Care 2005;14:77-78.
FULL TEXT  

Relationship between accreditation scores and the public disclosure of accreditation reports: a cross sectional study
Ito and Sugawara
Qual Saf Health Care 2005;14:87-92.
ABSTRACT | FULL TEXT  

Condition based payment: improving care of chronic illness
DiPiero and Sanders
BMJ 2005;330:654-657.
FULL TEXT  

The Unintended Consequences of Publicly Reporting Quality Information
Werner and Asch
JAMA 2005;293:1239-1244.
ABSTRACT | FULL TEXT  

Creating A State Minority Health Policy Report Card
Trivedi et al.
Health Aff (Millwood) 2005;24:388-396.
ABSTRACT | FULL TEXT  

Report Cards and Nursing Homes
Castle and Lowe
Gerontologist 2005;45:48-67.
ABSTRACT | FULL TEXT  

The Influence of Public Reporting of Outcome Data on Medical Decision Making by Physicians
Narins et al.
Arch Intern Med 2005;165:83-87.
ABSTRACT | FULL TEXT  

Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example
Huang et al.
Med Decis Making 2005;25:20-34.
ABSTRACT  

Response to "Patient organisations should also establish databanks on medical complications"
Marang-van de Mheen and Kievit
J. Med. Ethics 2004;30:609-610.
FULL TEXT  

Hospital Report Cards: Intent, Impact, and Illusion
Rosenstein
American Journal of Medical Quality 2004;19:183-192.
ABSTRACT  

Variations In The Quality Of Care For Very-Low-Birthweight Infants: Implications For Policy
Rogowski et al.
Health Aff (Millwood) 2004;23:88-97.
ABSTRACT | FULL TEXT  

The legacy of Bristol: public disclosure of individual surgeons' results
Keogh et al.
BMJ 2004;329:450-454.
FULL TEXT  

A Middle Ground on Public Accountability
Lee et al.
NEJM 2004;350:2409-2412.
FULL TEXT  

Patients' Interest in Reading Their Medical Record: Relation With Clinical and Sociodemographic Characteristics and Patients' Approach to Health Care
Fowles et al.
Arch Intern Med 2004;164:793-800.
ABSTRACT | FULL TEXT  

Physician performance information and consumer choice: a survey of subjects with the freedom to choose between doctors
Cheng and Song
Qual Saf Health Care 2004;13:98-101.
ABSTRACT | FULL TEXT  

Data feedback efforts in quality improvement: lessons learned from US hospitals
Bradley et al.
Qual Saf Health Care 2004;13:26-31.
ABSTRACT | FULL TEXT  

Organizing Regional Perinatal Quality Improvement: Global Considerations and Local Implementation
Wirtschafter and Powers
NeoReviews 2004;5:e50-59.
FULL TEXT  

Indirect vs Direct Hospital Quality Indicators for Very Low-Birth-Weight Infants
Rogowski et al.
JAMA 2004;291:202-209.
ABSTRACT | FULL TEXT  

What Will Become Of The Medical Mecca? Health Care Spending In Massachusetts
Mechanic
Health Aff (Millwood) 2003;22:130-141.
ABSTRACT | FULL TEXT  

Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement
Arah et al.
Int J Qual Health Care 2003;15:377-398.
ABSTRACT | FULL TEXT  

Using hierarchical models to analyse clinical indicators: a comparison of the gamma-Poisson and beta-binomial models
HOWLEY and GIBBERD
Int J Qual Health Care 2003;15:319-329.
ABSTRACT | FULL TEXT  

Public views on healthcare performance indicators and patient choice
Magee et al.
JRSM 2003;96:338-342.
ABSTRACT | FULL TEXT  

Boosting performance measure for measure
Stryer and Clancy
BMJ 2003;326:1278-1279.
FULL TEXT  

Public Reporting On Quality In The United States And The United Kingdom
Marshall et al.
Health Aff (Millwood) 2003;22:134-148.
ABSTRACT | FULL TEXT  

Primary care in the United States: Profiling performance in primary care in the United States
Goldfield et al.
BMJ 2003;326:744-747.
FULL TEXT  

Quality or financing: what drives design of the health care system?
McLoughlin and Leatherman
Qual Saf Health Care 2003;12:136-142.
ABSTRACT | FULL TEXT  

The Quality of Quality Measurement in U.S. Nursing Homes
Mor et al.
Gerontologist 2003;43:37-46.
ABSTRACT | FULL TEXT  

Impact of an acute myocardial infarction report card in Ontario, Canada
TU and CAMERON
Int J Qual Health Care 2003;15:131-137.
ABSTRACT | FULL TEXT  

Reporting quality of nursing home care to consumers: the Maryland experience
MATTKE et al.
Int J Qual Health Care 2003;15:169-177.
ABSTRACT | FULL TEXT  

Employers' Efforts To Measure And Improve Hospital Quality: Determinants Of Success
Mehrotra et al.
Health Aff (Millwood) 2003;22:60-71.
ABSTRACT | FULL TEXT  

Consumers And Quality-Driven Health Care: A Call To Action
Shaller et al.
Health Aff (Millwood) 2003;22:95-101.
ABSTRACT | FULL TEXT  

International comparisons of the quality of health care: what do they tell us?
Walshe
Qual Saf Health Care 2003;12:4-5.
FULL TEXT  

Safe treatment or a shorter wait: Hobson's choice?
Coulter
Qual Saf Health Care 2003;12:5-6.
FULL TEXT  

Medicare Quality Improvement: Bad Apples or Bad Systems?
Hsia
JAMA 2003;289:354-356.
FULL TEXT  

Risk Stratification and Comorbidity
Ferraris and Ferraris
Card Surg Adult 2003;2:187-224.
FULL TEXT  

Attitudes to the public release of comparative information on the quality of general practice care: qualitative study
Marshall et al.
BMJ 2002;325:1278-1278.
ABSTRACT | FULL TEXT  

Volume of procedures and outcome of treatment
Soljak
BMJ 2002;325:787-788.
FULL TEXT  

Relationship Between Low Quality-of-Care Scores and HMOs' Subsequent Public Disclosure of Quality-of-Care Scores
McCormick et al.
JAMA 2002;288:1484-1490.
ABSTRACT | FULL TEXT  

Public Performance Reports and the Will for Change
Berwick
JAMA 2002;288:1523-1524.
FULL TEXT  

Large Employers' New Strategies in Health Care
Galvin and Milstein
NEJM 2002;347:939-942.
FULL TEXT  

Analytic Reviews: Considerations in the Development of Intensive Care Unit Report Cards
Gupta et al.
J Intensive Care Med 2002;17:211-217.
ABSTRACT  

Effect of CAHPS Performance Information on Health Plan Choices by Iowa Medicaid Beneficiaries
Farley et al.
Med Care Res Rev 2002;59:319-336.
ABSTRACT  

Health Care Report Cards: Implications for Vulnerable Patient Groups and the Organizations Providing Them Care
Davies et al.
Journal of Health Politics, Policy and Law 2002;27:379-400.
ABSTRACT  

Volume and Outcome -- It Is Time to Move Ahead
Epstein
NEJM 2002;346:1161-1164.
FULL TEXT  

Public Profiling of Clinical Performance
Naylor
JAMA 2002;287:1323-1325.
FULL TEXT  

What Cognitive Science Tells Us about the Design of Reports for Consumers
Vaiana and McGlynn
Med Care Res Rev 2002;59:3-35.
ABSTRACT