 |
|
The Public Release of Performance Data
What Do We Expect to Gain? A Review of the Evidence
Martin N. Marshall, MSc, MD, FRCGP;
Paul G. Shekelle, MD, PhD;
Sheila Leatherman, MSW;
Robert H. Brook, MD, ScD
JAMA. 2000;283:1866-1874.
ABSTRACT
| |
Context Information about the performance of hospitals, health professionals, and health care organizations has been made public in the United States for more than a decade. The expected gains of public disclosure have not been made clear, and both the benefits and potential risks have received minimal empirical investigation.
Objective To summarize the empirical evidence concerning public disclosure of performance data, relate the results to the potential gains, and identify areas requiring further research.
Data Sources A literature search was conducted on MEDLINE and EMBASE databases for articles published between January 1986 and October 1999 in peer-reviewed journals. Review of citations, public documents, and expert advice was conducted to identify studies not found in the electronic databases.
Study Selection Descriptive, observational, or experimental evaluations of US reporting systems were selected for inclusion.
Data Extraction Included studies were organized based on use of public data by consumers, purchasers, physicians, and hospitals; impact on quality of care outcomes; and costs.
Data Synthesis Seven US reporting systems have been the subject of published empirical evaluations. Descriptive and observational methods predominate. Consumers and purchasers rarely search out the information and do not understand or trust it; it has a small, although increasing, impact on their decision making. Physicians are skeptical about such data and only a small proportion makes use of it. Hospitals appear to be most responsive to the data. In a limited number of studies, the publication of performance data has been associated with an improvement in health outcomes.
Conclusions There are several potential gains from the public disclosure of performance data, but use of the information by provider organizations for quality improvement may be the most productive area for further research.
INTRODUCTION
Information about the performance of hospitals, health professionals, and health care organizations is increasingly being released into the public domain.1 The data, often produced in the form of "report cards," "provider profiles," or "consumer reports," necessitates the development and dissemination of standardized reports on quality of care and facilitates comparisons of performance over time, among providers, and against defined standards of good practice. Health care performance data have been made public in the United States for more than a decade,2 and the production and dissemination of report cards is now a multimillion-dollar industry. However, evaluation of the impact of report cards has not kept pace with the development of reporting systems.3-7 In addition, there has been minimal agreement among the various stakeholders about expected gains from the release of comparative performance data. It has therefore proved difficult to judge whether the benefits of public disclosure of performance data outweigh the disadvantages.
The failure to articulate a clear rationale for the public release of performance data is illustrated by the wide variety of ways in which commentators believe the data might be used. The most common expectation is that such data will promote an efficient market economy in health care,8-9 usually in the belief that information about performance will encourage consumers to choose to access high-quality providers.10-13 Some authorities suggest that the information could be used by providers as a marketing tool.14 Others suggest that it will help to control costs15-16 or at least counter the influence of cost as the principal determinant of purchaser decision making.17-19 In addition, public information about performance has been proposed as a tool to regulate the health system,10, 20 a method of ensuring accountability of provider organizations,14, 21 or of making judgments about the performance of individual professionals.4 Finally, some perceive public disclosure to be a mechanism to promote quality improvement by informing purchasers18 or encouraging providers to focus on quality problems.10, 14, 22-23
Public disclosure therefore has been advocated as a mechanism to regulate providers of care, ensure accountability, inform, promote quality improvement, and encourage cost control. These diverse aims have made it difficult to identify evaluation criteria that can be used to assess its impact. In this article, we examine the research evidence to determine the extent to which the expectations of public disclosure have been realized and identify a future research agenda.
METHODS
Data Sources, Study Selection, and Data Extraction
An extensive review of published articles in peer-reviewed journals was conducted between October 1998 and October 1999 using MEDLINE and EMBASE electronic databases. We limited our review to evaluations of US reporting systems published between January 1986 and October 1999. While organizations may carry out evaluations without publishing the results, we considered peer-review publication an important marker of the rigor of such evaluations and therefore restricted our search to published reports. Searches were conducted independently by 1 of the authors (M.N.M.) and a professional librarian using the following Medical Subject Headings: report cards, public performance reports, provider profiling, public/consumer/patient information, and consumer reports. Only original research articles were reviewed. The citations in all articles were searched. Authors of the principal published studies and other experts in the field were asked to recommend relevant published studies. In addition, documents and Web sites prepared by a range of public and private organizations were searched for references to original evaluative studies relating to public disclosure.
Data Synthesis
There are many reporting systems in the United States but only 7 of these systems appear to have been subjected to rigorous empirical evaluation. A total of 21 peer-reviewed publications were found relating to these 7 systems (Table 1). Descriptive, observational, and quasi-experimental study designs were most commonly used; there was only 1 randomized controlled trial. For the purpose of this review, the literature has been categorized into use of public reports by the key stakeholders (consumers, purchasers, physicians, and provider organizations), the impact on quality of care outcomes, and the financial costs (Table 2). The term "provider" refers principally to hospitals in the studies. The studies cited in Table 1 that are relevant to the data extraction categories are described in greater detail in Table 2. In addition to these 21 studies, 5 other articles are cited that describe general issues relating to public disclosure but are not specific to any 1 reporting system.
|
|
|
|
Table 1. Reporting Systems Evaluated in Peer-Reviewed Publications*
|
|
|
|
|
|
|
Table 2. Summary of Evidence*
|
|
|
Do Consumers Use Public Reports?
There is a growing body of evidence that consumers want more information about health care provider performance and are able and willing to identify the content and format of the information of greatest use to them.9, 36 However, evidence suggests that the information has only a limited impact on consumer decision making.
Consumer use of the Pennsylvania report card, which publishes hospital- and surgeon-specific data on coronary artery bypass graft (CABG) mortality, was assessed using a telephone survey of patients who had undergone CABG surgery in the previous year in 4 Pennsylvania hospitals.12 Two of the hospitals had lower and 1 higher than expected mortality rates. Of 474 patients surveyed, 56 were aware of the performance report at the time of their surgery and less than one quarter of these stated that it had any significant impact on their choice of surgeon. Most of these patients were unable to correctly specify the data on which they based their choice of surgeon. When the report was described, more than half of the respondents said that the information would have influenced their choice of surgeon, but there was a low level of willingness to pay for it.
Hibbard and Weeks11 studied the impact of comparative data about physician fees on consumer use of services. The authors randomly allocated Medicare enrollees and state employees in Oregon to receive information about physician fees. No statistically significant difference in physician visit rate, expenditures on ambulatory care, or costs per visit was observed between the intervention and control groups. Other studies have shown that consumers continue to use hospitals with high mortality rates27-28 and that willingness to use hospitals is more likely to be influenced by anecdotal press reports of untoward deaths in hospitals than by risk-adjusted mortality data.28
The only study suggesting that public disclosure had a significant effect on consumer decision making evaluated the impact of the New York State CABG mortality report on hospital and physician market share and price changes before and after publication of the data.19 This quasi-experimental study indicated that hospitals and physicians with better outcomes experienced higher rates of growth in market share and that physicians with better outcomes had higher rates of growth of charges for the procedure. The magnitude of the association declined over time, suggesting that the market responded primarily to new information.
The reasons for consumers' lack of interest in and use of performance data include difficulty in understanding the information,35, 37 disinterest in the nature of the information available,39 lack of trust in the data,36 problems with timely access to the information,12 and lack of choice.40 There is evidence from descriptive studies that consumers rate anecdotal evidence from family and friends more highly than empirical evidence.28, 36
Do Purchasers Use Public Reports?
Employers are the main purchasers of health care in the United States, both as individual corporations and in coalitions that have considerable aggregate purchasing power. Policy analysts have expected these coalitions to be more responsive to public performance data than individual consumers, but current evidence suggests that public disclosure has only a small, although possibly increasing, effect on purchasing behavior.
The National Committee for Quality Assurance (NCQA) is responsible for 1 of the highest-profile reporting systems in the United States, the Health Employer Data Information Set (HEDIS) and the NCQA accreditation program. Its impact on employer choice of plan was examined by surveying large US employers.29 The results were compared with a similar survey conducted by the same research team the previous year. The percentage of employers familiar with accreditation increased from 29% to 35%. Of employers, 11% considered NCQA accreditation to be very important, and 5% considered HEDIS data to be very important. Less than 10% required NCQA accreditation for plan selection and 1% provided the HEDIS data to their employees to help them choose a plan. Only marginally greater use of performance data was found in a similar study by Hibbard et al.18
Do Physicians Use Public Reports?
The acceptability to and use of public performance data by physicians may be an important factor in the successful implementation of a public disclosure policy. The evidence suggests that physicians are interested in but skeptical about such data and that they consider it to be of minimal use.
A survey of cardiologists' attitudes to and use of the New York State CABG mortality data revealed that most cardiologists found the reports easy to read and considered the data to be very or somewhat accurate.32 Twenty-two percent routinely discussed the data with their patients, and 38% considered it to have influenced their referral pattern. The most common objection to the reports was a perception that it discouraged cardiac surgeons from operating on high-risk patients.
A survey of cardiologists' and cardiac surgeons' responses to the Pennsylvania CABG reporting system produced similar results23—a minority considered the data to be important when assessing the performance of a particular surgeon or discussed the data with their patients. Almost two thirds of cardiologists reported increasing problems finding surgeons to operate on high-risk patients, and the same proportion of cardiac surgeons reported that they were less willing to operate on such patients.
Do Hospitals and Other Provider Organizations Use Public Reports?
There is some evidence that provider organizations are more responsive to performance data than consumers, purchasers, or individual physicians.
A quasi-experimental design was used to examine the impact of an obstetrics consumer report on hospital behavior in Missouri.14 Half of the hospitals that did not have an infant car seat program, formal transfer arrangements, or breastfeeding nurse educators prior to publication of the report instituted or planned these services after the report was published. Hospitals in competitive markets were twice as likely to implement changes than those with monopolies. Several clinical outcome indicators improved after publication, including patient satisfaction and cesarean delivery rates.
Studies conducted in organizations exposed to publication of data about their performance have produced mixed assessments of the use of the information. Some organizations, particularly those whose performance is shown in a positive light by the reports, use the information for benchmarking, to promote collaboration across departments, and for internal monitoring of performance.8, 20, 33 In contrast, other organizations respond more negatively, particularly in the early stages of release of the data.15, 22, 24-25 Providers that have been identified as poor performers are more likely to criticize the validity of the data and the resource implications of acting on it.24
Impact on Quality of Care Outcomes
The most fundamental question relating to public disclosure is the extent to which it influences outcomes of care. Three studies specifically attempted to address this issue, all of which used quasi-experimental designs.
Hannan et al10 studied all patients undergoing isolated CABG surgery who were discharged between 1989 and 1992 from New York hospitals where the procedure was performed. A clinical database was used to identify significant independent risk factors. Actual mortality decreased by 21% during the study period. Illness severity of the patients being operated on increased during this time so that risk-adjusted mortality decreased even further, by 41%, which was considerably higher than the national average. The study occasioned considerable debate among academics, and several further studies attempted to support or refute the authors' conclusions that publication made a significant contribution to the observed improvement.31, 34, 41-44
Potential explanations for the New York data have been suggested31 and relate largely to changes in hospital policy resulting from public disclosure.30, 43 First, an exodus of low-volume and high-mortality surgeons followed the publication of the data, probably as a result of hospitals restricting operating privileges. Second, a marked improvement in the performance of non–low-volume surgeons was observed. Finally, there was an improvement in the performance of surgeons new to the system.
The main criticism of the release of data in New York was that the data could have reduced access to CABG surgery by forcing sicker patients to seek surgery outside the state or by surgeons refusing to operate on high-risk patients. Peterson et al34 used national Medicare data to examine trends in the percentage of New York residents receiving out-of-state surgery before and after initiation of the provider profiling program. They also examined procedure use by elderly patients with myocardial infarctions within the state to determine whether high-risk patients were being refused treatment. Contrary to a previous single-center study,41 Peterson and colleagues found that the percentage of New York residents receiving out-of-state bypass operations decreased between 1987 and 1992 and the likelihood of bypass surgery following myocardial infarctions increased. They confirmed a reduction in 30-day mortality in New York well above the national average during the period studied. The only other area with similar figures was northern New England, where provider profile reports were also provided, although for internal use rather than for publication.45
In an evaluation of the Cleveland Health Quality Choice project, Rosenthal and colleagues26 reported improved outcomes associated with publication of mortality data for 6 common medical conditions and 2 surgical operations. The authors examined 101,060 discharges from 30 northeastern Ohio hospitals and found significant and sustained reductions in risk-adjusted mortality for most conditions following publication.
Impact on Costs
There has been little evaluation of the costs of publishing performance data, in part because some of the data are collected anyway for internal evaluation and administrative purposes and because of the multiple contributors to data collection, analysis, and dissemination. The only published estimate that could be found described the costs of producing data for the Pennsylvania reporting system.38 Average costs were estimated to be $16 per patient discharge, at 1991 prices, to collect and report the data. Costs depend on the data system used, but given the initial costs of the development of measures, analytical methods, and data management systems and ongoing costs of data collection, analysis, auditing, dissemination, and management of the responses, this figure is likely to be an underestimate of costs at 2000 prices.
COMMENT
Public disclosure of performance data is a major health policy initiative that represents a potential challenge to health professionals' traditional concept of autonomy.46 It is therefore surprising that after more than a decade of experience of public disclosure in the United States, so few current reporting systems have been the subject of so little published empirical evaluation.
There are several possible reasons for this lack of examination. One is an inevitable time lag between the introduction and evaluation of a new initiative. In addition, some people may regard public disclosure as so obviously the right thing to do that formal evaluation of its impact is not necessary, or the "political correctness" of challenging a tool of informed consumerism may discourage candid appraisal. Also, an evaluation of current reporting systems may be being delayed until more sophisticated data are available, and vested business interests may feel threatened by formal evaluation of an area in which there is considerable business potential. Moreover, funding agencies may not see evaluation as a priority, and the methodological problems of researching such complex interventions may be a deterrent.7
Whether the limited evidence summarized in this article is regarded as supportive of, or counter to, a policy promoting greater public disclosure is open to debate. Those who are in favor will focus on the small but significant gains and will question the sensitivity of conventional research methods to measure subtle improvements in quality of care. They may also point out that the main impact of the information will not be seen for several years, particularly if the quality of the data continues to improve. Antagonists will focus on the negative findings and will emphasize the small gains in relation to the considerable public and private investment of time and resources.
Figure 1 summarizes the issues pertaining to public disclosure in a conceptual framework. The evidence does not provide strong support for those who believe that public disclosure should focus on the role of the consumer or purchaser. Neither individual consumers nor group purchasers appear to search out, understand, or use the currently available information to any significant extent. If consumers are regarded by some as a key audience, then performance data should perhaps be aimed more at consumer advocates and interest groups than at individual consumers. The skepticism of physicians suggests that they too are unlikely to be key players in promoting or encouraging more effective use of publicly available data. The key stakeholders seem to be the provider organizations because the available evidence suggests that publication of performance data results in provider behavior change that contributes to observed improvement in both the processes and outcomes of care. Provider organizations are sensitive to their public image and potential legal risk and often have the authority to act on suboptimal levels of performance and promote good standards of practice. They therefore respond to performance data as a competitive opportunity or risk management imperative even though the purchasing side of the market is less sensitive to it. An intrinsic professional competitiveness on the part of the provider organizations, based on a desire to be seen to be performing well, might therefore be just as important a motivator as economic gain. If this is the case, the reasons for organizational providers being more sensitive to the data than individual physicians warrants further investigation.
|
|
|
|
Figure. Conceptual Model of Public Disclosure
|
|
|
This review of the published research reveals many unanswered questions. First, while few would question the importance of performance data as one component of quality improvement, there is considerable uncertainty about the relative merits and risks of using such data internally or making it publicly available. Second, the unintended and potentially adverse consequences of public disclosure have received minimal attention. There is anecdotal evidence that making performance data public can result in gaming and an inappropriate focus on what is being measured.47-48 The negative impact of publication of poor performance on public trust and professional morale6 has also been highlighted, but these issues require empirical investigation. Third, there is considerable doubt about the most effective and appropriate level for the reporting of performance data and the degree of risk adjustment of health outcomes required to achieve a balance between cost, effectiveness, and fairness to providers.49-50
The philosophical stance of those who evaluate the impact of public disclosure should be explicit. The narrowest perspective, and the one that has been most commonly adopted, would view public disclosure as a specific intervention and consider only those benefits and costs that relate directly to release of the data. A broader perspective, analogous to other examples relating to freedom of information, would consider implications for society, such as benefits arising from public discussion, damage to professional reputations, and resource implications resulting from demands to see the "best" provider. The broadest perspective would consider disclosure as a right in a democratic society, irrespective of the costs, and focus on release of data as an intended outcome, rather than a controllable intervention.
An increasing level of public disclosure of health care performance data is inevitable and reflects greater demands for information and accountability in all walks of life. We believe that the use of public performance data by consumers and purchasers or for regulation purposes, will remain relatively less important for the foreseeable future than use of the data as a catalyst to stimulate and promote internal quality improvement mechanisms at the level of the organizational provider. Articulation of such a clear purpose for public disclosure and the development of an evidence base to guide its implementation will ensure that disclosure is conducted in a way that will maximize the potential benefits and reduce the associated risks.
AUTHOR INFORMATION
Author Contributions: Dr Brook devised the project in association with Drs Marshall and Shekelle. Dr Marshall conducted the literature review and wrote the first draft of the paper. All authors contributed to the ideas and subsequent redrafts of the paper. Dr Marshall is the guarantor.
Funding/Support: This study was supported by the Nuffield Trust, London, England, which commissioned and funded the work. Dr Marshall was a Harkness Fellow in Health Care Policy, based at RAND, Santa Monica, Calif, while researching and writing the report. The Fellowship program is supported by the Commonwealth Fund, New York, NY.
Disclaimer: The views expressed here are those of the authors and not necessarily those of the Nuffield Trust, Commonwealth Fund, or Veterans Affairs Health Service.
Acknowledgment: We thank the individuals and organizations who agreed to talk about their experiences of public disclosure and gave generously of their time and expertise.
Corresponding Author and Reprints: Martin N. Marshall, MSc, MD, FRCGP, Senior Research Fellow, National Primary Care Research and Development Centre, University of Manchester, Fifth Floor, Williamson Bldg, Oxford Road, Manchester M13 9PL, UK (e-mail: mmarshall{at}fs1.cpcr.man.ac.uk).
Author Affiliations: School of Postgraduate Medicine and Health Sciences, University of Exeter, Exeter, England (Dr Marshall); RAND Health Program, Santa Monica (Drs Shekelle and Brook), Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles (Dr Shekelle), Calif; Center for Health Care Policy and Evaluation, United Health Group, Minnetonka, Minn (Ms Leatherman); Judge Institute of Management, University of Cambridge, Cambridge, England (Ms Leatherman); and University of California, Los Angeles, Center for Health Services, Los Angeles (Dr Brook). Dr Marshall is now with the National Primary Care Research and Development Centre, University of Manchester, Manchester, England.
REFERENCES
| |
1. Davies HT, Marshall MN. Public disclosure of performance data. Lancet. 1999;353:1639-1640.
FULL TEXT
|
ISI
| PUBMED
2. Epstein A. Rolling down the runway: the challenges ahead for quality report cards. JAMA. 1998;279:1691-1696.
FREE FULL TEXT
3. Green J, Wintfield N. Report cards on cardiac surgeons. N Engl J Med. 1995;332:1229-1232.
FREE FULL TEXT
4. Kassirer JP. The use and abuse of practice profiles. N Engl J Med. 1994;330:634-635.
FREE FULL TEXT
5. Lansky D. Overview: performance measures. Jt Comm J Qual Improv. 1996;22:439-442.
PUBMED
6. Davies HT, Lampel J. Trust in performance indicators. Qual Health Care. 1998;7:159-162.
ABSTRACT
7. Thomson R, Lally J. Clinical indicators. Qual Health Care. 1998;7:122.
ISI
| PUBMED
8. Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery. Jt Comm J Qual Improv. 1998;24:40-49.
PUBMED
9. Edgman-Levitan S, Cleary PD. What information do consumers want and need? Health Aff (Millwood). 1996;15:42-56.
PUBMED
10. Hannan EL, Kilburn H, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA. 1994;271:761-766.
FREE FULL TEXT
11. Hibbard JH, Weeks EC. Does the dissemination of comparative data on physician fees affect consumer use of services? Med Care. 1989;27:1167-1174.
FULL TEXT
|
ISI
| PUBMED
12. Schneider EC, Epstein AM. Use of public performance reports. JAMA. 1998;279:1638-1642.
FREE FULL TEXT
13. Lansky D. Measuring what matters to the public. Health Aff (Millwood). 1998;17:40-41.
PUBMED
14. Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care. JAMA. 1997;278:1579-1584.
FREE FULL TEXT
15. Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data. JAMA. 1990;263:247-249.
FREE FULL TEXT
16. Sirio CA, McGee JL. Public reporting of clinical outcomes. Am J Med Qual. 1996;11:S78-S81.
17. Brook RH. Health reform is on its way. Ann Intern Med. 1994;120:84-86.
FREE FULL TEXT
18. Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan. Health Aff (Millwood). 1997;16:172-180.
ABSTRACT
19. Mukamel DB, Mushlin AI. Quality of care information makes a difference. Med Care. 1998;36:945-954.
FULL TEXT
|
ISI
| PUBMED
20. Rosenthal GE, Hammar PJ, Way LE, et al. Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience. Jt Comm J Qual Improv. 1998;24:347-360.
PUBMED
21. Topol EJ, Califf RM. Scorecard cardiovascular medicine: its impact and future directions. Ann Intern Med. 1994;120:65-70.
FREE FULL TEXT
22. Rainwater JA, Romano PS, Antonius DM. The California Hospital Outcomes Project. Jt Comm J Qual Improv. 1998;24:31-39.
PUBMED
23. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. N Engl J Med. 1996;335:251-256.
FREE FULL TEXT
24. Luce JM, Thiel GD, Holland MR, et al. Use of risk-adjusted outcome data for quality improvement by public hospitals. West J Med. 1996;164:410-414.
ISI
| PUBMED
25. Romano PS, Rainwater JA, Antonius D. Grading the graders: how hospitals in California and New York perceive and interpret their report cards. Med Care. 1999;37:295-305.
FULL TEXT
|
ISI
| PUBMED
26. Rosenthal GE, Quinn L, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance. Am J Med Qual. 1997;12:103-112.
FREE FULL TEXT
27. Vladeck BC, Goodwin EJ, Myers LP, Sinisi M. Consumers and hospital use: the HCFA "death list." Health Aff (Millwood). 1988;7:122-125.
FULL TEXT
| PUBMED
28. Mennemeyer ST, Morrisey MA, Howard LZ. Death and reputation: how consumers acted upon HCFA mortality information. Inquiry. 1997;34:117-128.
ISI
| PUBMED
29. Gabel JR, Hunt KA, Hurst K, Peat Marwick KP. When Employers Choose Health Plans. New York, NY: Commonwealth Fund; 1998. Report 293.
30. Hannan EL, Kumar D, Racz M, et al. New York State's Cardiac Surgery Reporting System. Ann Thorac Surg. 1994;58:1852-1857.
ABSTRACT
31. Hannan EL, Siu AL, Kumar D, Kilburn H, Chassin MR. The decline in coronary artery bypass graft mortality in New York State. JAMA. 1995;273:209-213.
FREE FULL TEXT
32. Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York. Am Heart J. 1997;134:55-61.
FULL TEXT
|
ISI
| PUBMED
33. Dziuban SW, McIlduff JB, Miller SJ, Dal Col RH. How a New York cardiac surgery program uses outcomes data. Ann Thorac Surg. 1994;58:1871-1876.
ABSTRACT
34. Peterson ED, Delong ER, Jollis JG, Muhlbaier LH, Mark DB. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. J Am Coll Cardiol. 1998;32:993-999.
FREE FULL TEXT
35. Jewett JJ, Hibbard JH. Comprehension of quality of care indicators. Health Care Financ Rev. 1996;18:75-94.
ISI
| PUBMED
36. Robinson S, Brodie M. Understanding the quality challenge for health consumers: the Kaiser/AHCPR survey. Jt Comm J Qual Improv. 1997;23:239-244.
PUBMED
37. Hibbard JH, Jewett JJ, Engelmann S, Tusler M. Can Medicare beneficiaries make informed choices? Health Aff (Millwood). 1998;17:181-193.
PUBMED
38. US General Accounting Office. Health Care Reform: "Report Cards" Are Useful but Significant Issues Need to Be Addressed. Washington, DC: US General Accounting Office; 1994. Publication HEHS-94-219.
39. Goldfield N, Larson C, Roblin D, et al. The content of report cards. Jt Comm J Qual Improv. 1999;25:423-433.
40. Schoen C, Davis K. Erosion of Employer-Sponsored Health Insurance Coverage and Quality [periodic brief]. New York, NY: Commonwealth Fund; 1998.
41. Omoigui NA, Miller DP, Brown KJ, et al. Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes. Circulation. 1996;93:27-33.
FREE FULL TEXT
42. Hannan EL, Siu AL, Kumar D, et al. Assessment of coronary artery bypass graft surgery performance in New York. Med Care. 1997;35:49-56.
FULL TEXT
|
ISI
| PUBMED
43. Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. N Engl J Med. 1996;334:394-398.
FREE FULL TEXT
44. Ghali WA, Ash AS, Hall RE, Moskowitz MA. Statewide quality improvement initiatives and mortality after cardiac surgery. JAMA. 1997;277:379-382.
FREE FULL TEXT
45. O'Connor GT, Plume SK, Olmstead EM, Morton JR, Maloney CT. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. JAMA. 1996;275:841-846.
FREE FULL TEXT
46. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The Publication of Performance Data in Health Care. London, England: Nuffield Trust. In press.
47. Goddard M, Mannion R, Smith PC. The NHS Performance Framework. York, England: Centre for Health Economics; 1998. The York series on the National Health Service White Paper, No. 158.
48. Smith P. On the unintended consequences of publishing performance data in the public sector. Int J Public Adm. 1995;18:277-310.
49. Iezzoni LI. The risks of risk adjustment. JAMA. 1997;278:1600-1607.
FREE FULL TEXT
50. Iezzoni LI. Risk Adjustment for Measuring Health Care Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati
What's this?
RELATED LETTER
Use of Performance Data to Change Physician Behavior
Alan H. Rosenstein, Martin N. Marshall, Paul G. Shekelle, Robert H. Brook, and Sheila Leatherman
JAMA. 2000;284(9):1079.
EXTRACT
| FULL TEXT
RELATED ARTICLES
Public Release of Performance Data: A Progress Report From the Front
Arnold M. Epstein
JAMA. 2000;283(14):1884-1886.
EXTRACT
| FULL TEXT
April 12, 2000
JAMA. 2000;283(14):1905-1906.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Is There Evidence of Cream Skimming Among Nursing Homes Following the Publication of the Nursing Home Compare Report Card?
Mukamel et al.
The Gerontologist 2009;0:gnp062v1-gnp062.
ABSTRACT
| FULL TEXT
2009 Focused Update Incorporated Into the ACC/AHA 2005 Guidelines for the Diagnosis and Management of Heart Failure in Adults: A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines Developed in Collaboration With the International Society for Heart and Lung Transplantation
Hunt et al.
J Am Coll Cardiol 2009;53:e1-e90.
FULL TEXT
2009 Focused Update Incorporated Into the ACC/AHA 2005 Guidelines for the Diagnosis and Management of Heart Failure in Adults: A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines: Developed in Collaboration With the International Society for Heart and Lung Transplantation
2005 WRITING COMMITTEE MEMBERS et al.
Circulation 2009;119:e391-e479.
FULL TEXT
Communicating Hospital Infection Data to the Public: A Study of Consumer Responses and Preferences
Mazor et al.
American Journal of Medical Quality 2009;24:108-115.
ABSTRACT
Motivating Public Use of Physician-Level Performance Data: An Experiment on the Effects of Message and Mode
Ranganathan et al.
Med Care Res Rev 2009;66:68-81.
ABSTRACT
Contextualization of Physical and Sexual Assault in Male Prisons: Incidents and Their Aftermath
Wolff and Jing Shi
J Correct Health Care 2009;15:58-77.
ABSTRACT
Public Reporting of Quality Data for Stroke: Is It Measuring Quality?
Kelly et al.
Stroke 2008;39:3367-3371.
ABSTRACT
| FULL TEXT
Nonprimary Care Physicians' Views on Office-Based Quality Incentive and Improvement Programs
Murphy and Nash
American Journal of Medical Quality 2008;23:427-439.
ABSTRACT
Regulation, measurements and incentives. The experience in the US and UK: does context matter?
Hamblin
The Journal of the Royal Society for the Promotion of Health 2008;128:291-298.
ABSTRACT
Choosing The Best Hospital: The Limitations Of Public Quality Reporting
Rothberg et al.
Health Aff (Millwood) 2008;27:1680-1687.
ABSTRACT
| FULL TEXT
Willingness of patients to change surgeons for a shorter waiting time for joint arthroplasty
Conner-Spady et al.
CMAJ 2008;179:327-332.
ABSTRACT
| FULL TEXT
Is choice working for patients in the English NHS?
Bevan
BMJ 2008;337:a935-a935.
FULL TEXT
What benefits will choice bring to patients? Literature review and assessment of implications
Fotaki et al.
J Health Serv Res Policy 2008;13:178-184.
ABSTRACT
| FULL TEXT
ACCF 2008 Health Policy Statement on Principles for Public Reporting of Physician Performance Data: A Report of the American College of Cardiology Foundation Writing Committee to Develop Principles for Public Reporting of Physician Performance Data
Drozda et al.
J Am Coll Cardiol 2008;51:1993-2001.
FULL TEXT
Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care
Fung et al.
ANN INTERN MED 2008;148:111-123.
ABSTRACT
| FULL TEXT
Defining poor and optimum performance in an IVF programme
Castilla et al.
Hum Reprod 2008;23:85-90.
ABSTRACT
| FULL TEXT
Risk Stratification and Comorbidity
Ferraris et al.
Card Surg Adult 2008;3:199-246.
FULL TEXT
Predictors of Timely Follow-Up Care Among Medicaid-Enrolled Adults After Psychiatric Hospitalization
Stein et al.
Psychiatr. Serv. 2007;58:1563-1569.
ABSTRACT
| FULL TEXT
Promoting quality: the health-care organization from a management perspective
Glickman et al.
Int J Qual Health Care 2007;19:341-348.
ABSTRACT
| FULL TEXT
Do we know quality epilepsy care when we see it?
Langfitt
Neurology 2007;69:1968-1969.
FULL TEXT
Improving the quality of care with performance indicators
Majeed et al.
BMJ 2007;335:916-918.
FULL TEXT
Publicly Available Hospital Comparison Web Sites: Determination of Useful, Valid, and Appropriate Information for Comparing Surgical Quality
Leonardi et al.
Arch Surg 2007;142:863-869.
ABSTRACT
| FULL TEXT
Publishing 'quality' measures: how it works and when it does not?
Hamblin
Int J Qual Health Care 2007;19:183-186.
FULL TEXT
Relationship between Number of Medical Conditions and Quality of Care
Higashi et al.
NEJM 2007;356:2496-2504.
ABSTRACT
| FULL TEXT
A comparison of quality of care indicators in urban acute care hospitals and rural critical access hospitals in the United States
Lutfiyya et al.
Int J Qual Health Care 2007;19:141-149.
ABSTRACT
| FULL TEXT
Choosing Where to Have Major Surgery: Who Makes the Decision?
Wilson et al.
Arch Surg 2007;142:242-246.
ABSTRACT
| FULL TEXT
Public Reporting and Pay for Performance in Hospital Quality Improvement
Lindenauer et al.
NEJM 2007;356:486-496.
ABSTRACT
| FULL TEXT
Measuring Performance For Treating Heart Attacks And Heart Failure: The Case For Outcomes Measurement
Krumholz et al.
Health Aff (Millwood) 2007;26:75-85.
ABSTRACT
| FULL TEXT
Relationship Between Medicare's Hospital Compare Performance Measures and Mortality Rates
Werner and Bradlow
JAMA 2006;296:2694-2702.
ABSTRACT
| FULL TEXT
The Impact of Unmeasured Clinical Variables on the Accuracy of Hospital Report Cards: A Monte Carlo Study
Austin
Med Decis Making 2006;26:447-466.
ABSTRACT
The Impact Of Quality-Reporting Programs On Hospital Operations
Pham et al.
Health Aff (Millwood) 2006;25:1412-1422.
ABSTRACT
| FULL TEXT
Do Cardiac Surgery Report Cards Reduce Mortality? Assessing the Evidence
Epstein
Med Care Res Rev 2006;63:403-426.
ABSTRACT
Is a public reporting approach appropriate for nursing home care?
Stevenson
Journal of Health Politics, Policy and Law 2006;31:773-810.
ABSTRACT
Paying for performance in the United States and abroad.
Epstein
NEJM 2006;355:406-408.
FULL TEXT
Strategies to Enhance Price and Quality Competition in Health Care: Lessons Learned from Tracking Local Markets
Lesser and Ginsburg
Journal of Health Politics, Policy and Law 2006;31:557-567.
ABSTRACT
Strategies for improving surgical quality--should payers reward excellence or effort?
Birkmeyer and Birkmeyer
NEJM 2006;354:864-870.
FULL TEXT
Reframing the Racial Disparities Issue for State Governments
Stone
Journal of Health Politics, Policy and Law 2006;31:127-152.
ABSTRACT
Using markets to reform health care
Edwards
BMJ 2005;331:1464-1466.
FULL TEXT
Regional Variation in Rates of Low Birth Weight
Thompson et al.
Pediatrics 2005;116:1114-1121.
ABSTRACT
| FULL TEXT
How do elderly patients decide where to go for major surgery? Telephone interview survey
Schwartz et al.
BMJ 2005;331:821.
ABSTRACT
| FULL TEXT
Quality of Care in U.S. Hospitals as Reflected by Standardized Measures, 2002-2004
Williams et al.
NEJM 2005;353:255-264.
ABSTRACT
| FULL TEXT
Administrative Data Feedback for Effective Cardiac Treatment: AFFECT, A Cluster Randomized Trial
Beck et al.
JAMA 2005;294:309-317.
ABSTRACT
| FULL TEXT
The Cost-Quality Trade-Off: Need for Data Quality Standards for Studies That Impact Clinical Practice and Health Policy
Malin and Keating
JCO 2005;23:4581-4584.
FULL TEXT
Hospital Performance Reports: Impact On Quality, Market Share, And Reputation
Hibbard et al.
Health Aff (Millwood) 2005;24:1150-1160.
ABSTRACT
| FULL TEXT
It Isn't Just about Choice: The Potential of a Public Performance Report to Affect the Public Image of Hospitals
Hibbard et al.
Med Care Res Rev 2005;62:358-371.
ABSTRACT
Improving the ICU: Part 2
Garland
Chest 2005;127:2165-2179.
ABSTRACT
| FULL TEXT
Fostering Rational Regulation of Patient Safety
Mello et al.
Journal of Health Politics, Policy and Law 2005;30:375-426.
ABSTRACT
Nursing Home Administrators' Opinions of the Nursing Home Compare Web Site
Castle
Gerontologist 2005;45:299-308.
ABSTRACT
| FULL TEXT
Impact of reporting hospital performance
Marshall and Romano
Qual Saf Health Care 2005;14:77-78.
FULL TEXT
Relationship between accreditation scores and the public disclosure of accreditation reports: a cross sectional study
Ito and Sugawara
Qual Saf Health Care 2005;14:87-92.
ABSTRACT
| FULL TEXT
Condition based payment: improving care of chronic illness
DiPiero and Sanders
BMJ 2005;330:654-657.
FULL TEXT
The Unintended Consequences of Publicly Reporting Quality Information
Werner and Asch
JAMA 2005;293:1239-1244.
ABSTRACT
| FULL TEXT
Creating A State Minority Health Policy Report Card
Trivedi et al.
Health Aff (Millwood) 2005;24:388-396.
ABSTRACT
| FULL TEXT
Report Cards and Nursing Homes
Castle and Lowe
Gerontologist 2005;45:48-67.
ABSTRACT
| FULL TEXT
The Influence of Public Reporting of Outcome Data on Medical Decision Making by Physicians
Narins et al.
Arch Intern Med 2005;165:83-87.
ABSTRACT
| FULL TEXT
Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example
Huang et al.
Med Decis Making 2005;25:20-34.
ABSTRACT
Response to "Patient organisations should also establish databanks on medical complications"
Marang-van de Mheen and Kievit
J. Med. Ethics 2004;30:609-610.
FULL TEXT
Hospital Report Cards: Intent, Impact, and Illusion
Rosenstein
American Journal of Medical Quality 2004;19:183-192.
ABSTRACT
Variations In The Quality Of Care For Very-Low-Birthweight Infants: Implications For Policy
Rogowski et al.
Health Aff (Millwood) 2004;23:88-97.
ABSTRACT
| FULL TEXT
The legacy of Bristol: public disclosure of individual surgeons' results
Keogh et al.
BMJ 2004;329:450-454.
FULL TEXT
A Middle Ground on Public Accountability
Lee et al.
NEJM 2004;350:2409-2412.
FULL TEXT
Patients' Interest in Reading Their Medical Record: Relation With Clinical and Sociodemographic Characteristics and Patients' Approach to Health Care
Fowles et al.
Arch Intern Med 2004;164:793-800.
ABSTRACT
| FULL TEXT
Physician performance information and consumer choice: a survey of subjects with the freedom to choose between doctors
Cheng and Song
Qual Saf Health Care 2004;13:98-101.
ABSTRACT
| FULL TEXT
Data feedback efforts in quality improvement: lessons learned from US hospitals
Bradley et al.
Qual Saf Health Care 2004;13:26-31.
ABSTRACT
| FULL TEXT
Organizing Regional Perinatal Quality Improvement: Global Considerations and Local Implementation
Wirtschafter and Powers
NeoReviews 2004;5:e50-59.
FULL TEXT
Indirect vs Direct Hospital Quality Indicators for Very Low-Birth-Weight Infants
Rogowski et al.
JAMA 2004;291:202-209.
ABSTRACT
| FULL TEXT
What Will Become Of The Medical Mecca? Health Care Spending In Massachusetts
Mechanic
Health Aff (Millwood) 2003;22:130-141.
ABSTRACT
| FULL TEXT
Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement
Arah et al.
Int J Qual Health Care 2003;15:377-398.
ABSTRACT
| FULL TEXT
Using hierarchical models to analyse clinical indicators: a comparison of the gamma-Poisson and beta-binomial models
HOWLEY and GIBBERD
Int J Qual Health Care 2003;15:319-329.
ABSTRACT
| FULL TEXT
Public views on healthcare performance indicators and patient choice
Magee et al.
JRSM 2003;96:338-342.
ABSTRACT
| FULL TEXT
Boosting performance measure for measure
Stryer and Clancy
BMJ 2003;326:1278-1279.
FULL TEXT
Public Reporting On Quality In The United States And The United Kingdom
Marshall et al.
Health Aff (Millwood) 2003;22:134-148.
ABSTRACT
| FULL TEXT
Primary care in the United States: Profiling performance in primary care in the United States
Goldfield et al.
BMJ 2003;326:744-747.
FULL TEXT
Quality or financing: what drives design of the health care system?
McLoughlin and Leatherman
Qual Saf Health Care 2003;12:136-142.
ABSTRACT
| FULL TEXT
The Quality of Quality Measurement in U.S. Nursing Homes
Mor et al.
Gerontologist 2003;43:37-46.
ABSTRACT
| FULL TEXT
Impact of an acute myocardial infarction report card in Ontario, Canada
TU and CAMERON
Int J Qual Health Care 2003;15:131-137.
ABSTRACT
| FULL TEXT
Reporting quality of nursing home care to consumers: the Maryland experience
MATTKE et al.
Int J Qual Health Care 2003;15:169-177.
ABSTRACT
| FULL TEXT
Employers' Efforts To Measure And Improve Hospital Quality: Determinants Of Success
Mehrotra et al.
Health Aff (Millwood) 2003;22:60-71.
ABSTRACT
| FULL TEXT
Consumers And Quality-Driven Health Care: A Call To Action
Shaller et al.
Health Aff (Millwood) 2003;22:95-101.
ABSTRACT
| FULL TEXT
International comparisons of the quality of health care: what do they tell us?
Walshe
Qual Saf Health Care 2003;12:4-5.
FULL TEXT
Safe treatment or a shorter wait: Hobson's choice?
Coulter
Qual Saf Health Care 2003;12:5-6.
FULL TEXT
Medicare Quality Improvement: Bad Apples or Bad Systems?
Hsia
JAMA 2003;289:354-356.
FULL TEXT
Risk Stratification and Comorbidity
Ferraris and Ferraris
Card Surg Adult 2003;2:187-224.
FULL TEXT
Attitudes to the public release of comparative information on the quality of general practice care: qualitative study
Marshall et al.
BMJ 2002;325:1278-1278.
ABSTRACT
| FULL TEXT
Volume of procedures and outcome of treatment
Soljak
BMJ 2002;325:787-788.
FULL TEXT
Relationship Between Low Quality-of-Care Scores and HMOs' Subsequent Public Disclosure of Quality-of-Care Scores
McCormick et al.
JAMA 2002;288:1484-1490.
ABSTRACT
| FULL TEXT
Public Performance Reports and the Will for Change
Berwick
JAMA 2002;288:1523-1524.
FULL TEXT
Large Employers' New Strategies in Health Care
Galvin and Milstein
NEJM 2002;347:939-942.
FULL TEXT
Analytic Reviews: Considerations in the Development of Intensive Care Unit Report Cards
Gupta et al.
J Intensive Care Med 2002;17:211-217.
ABSTRACT
Effect of CAHPS Performance Information on Health Plan Choices by Iowa Medicaid Beneficiaries
Farley et al.
Med Care Res Rev 2002;59:319-336.
ABSTRACT
Health Care Report Cards: Implications for Vulnerable Patient Groups and the Organizations Providing Them Care
Davies et al.
Journal of Health Politics, Policy and Law 2002;27:379-400.
ABSTRACT
Volume and Outcome -- It Is Time to Move Ahead
Epstein
NEJM 2002;346:1161-1164.
FULL TEXT
Public Profiling of Clinical Performance
Naylor
JAMA 2002;287:1323-1325.
FULL TEXT
What Cognitive Science Tells Us about the Design of Reports for Consumers
Vaiana and McGlynn
Med Care Res Rev 2002;59:3-35.
ABSTRACT
|
