This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (493)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Review  • Alert me on articles by topic  Social Bookmarking   What's this?

A Systematic Review

Dereck L. Hunt, MD; R. Brian Haynes, MD, PhD; Steven E. Hanna, MA, PhD; Kristina Smith

JAMA. 1998;280:1339-1346.

Context.— Many computer software developers and vendors claim that their systems can directly improve clinical decisions. As for other health care interventions, such claims should be based on careful trials that assess their effects on clinical performance and, preferably, patient outcomes.

Objective.— To systematically review controlled clinical trials assessing the effects of computer-based clinical decision support systems (CDSSs) on physician performance and patient outcomes.

Data Sources.— We updated earlier reviews covering 1974 to 1992 by searching the MEDLINE, EMBASE, INSPEC, SCISEARCH, and the Cochrane Library bibliographic databases from 1992 to March 1998. Reference lists and conference proceedings were reviewed and evaluators of CDSSs were contacted.

Study Selection.— Studies were included if they involved the use of a CDSS in a clinical setting by a health care practitioner and assessed the effects of the system prospectively with a concurrent control.

Data Extraction.— The validity of each relevant study (scored from 0-10) was evaluated in duplicate. Data on setting, subjects, computer systems, and outcomes were abstracted and a power analysis was done on studies with negative findings.

Data Synthesis.— A total of 68 controlled trials met our criteria, 40 of which were published since 1992. Quality scores ranged from 2 to10, with more recent trials rating higher (mean, 7.7) than earlier studies (mean, 6.4) (P<.001). Effects on physician performance were assessed in 65 studies and 43 found a benefit (66%). These included 9 of 15 studies on drug dosing systems, 1 of 5 studies on diagnostic aids, 14 of 19 preventive care systems, and 19 of 26 studies evaluating CDSSs for other medical care. Six of 14 studies assessing patient outcomes found a benefit. Of the remaining 8 studies, only 3 had a power of greater than 80% to detect a clinically important effect.

Conclusions.— Published studies of CDSSs are increasing rapidly, and their quality is improving. The CDSSs can enhance clinical performance for drug dosing, preventive care, and other aspects of medical care, but not convincingly for diagnosis. The effects of CDSSs on patient outcomes have been insufficiently studied.

From the Graduate Program in Health Research Methodology (Dr Hunt), the Health Information Research Unit, Department of Clinical Epidemiology and Biostatistics (Drs Hunt, Haynes, and Hanna and Ms Smith), and the Department of Medicine (Drs Hunt and Haynes), McMaster University Faculty of Health Sciences, Hamilton, Ontario.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati     What's this?

RELATED ARTICLES

Effect of Computerized Physician Order Entry and a Team Intervention on Prevention of Serious Medication Errors
David W. Bates, Lucian L. Leape, David J. Cullen, Nan Laird, Laura A. Petersen, Jonathan M. Teich, Elizabeth Burdick, Mairead Hickey, Sharon Kleefield, Brian Shea, Martha Vander Vliet, and Diane L. Seger
JAMA. 1998;280(15):1311-1316.
ABSTRACT | FULL TEXT  

A Computer Alert System to Prevent Injury From Adverse Drug Events: Development and Evaluation in a Community Teaching Hospital
Robert A. Raschke, Bea Gollihare, Thomas A. Wunderlich, James R. Guidry, Alan I. Leibowitz, John C. Peirce, Lee Lemelson, Mark A. Heisler, and Cynthia Susong
JAMA. 1998;280(15):1317-1320.
ABSTRACT | FULL TEXT  

Clinical Decision Support Systems to Improve Clinical Practice and Quality of Care
David C. Classen
JAMA. 1998;280(15):1360-1361.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Diabetes, kidney disease and cardiovascular disease patients. Assessing care of complex patients using outpatient testing and visits: additional metrics by which to evaluate health care system functioning
Levin et al.
Nephrol Dial Transplant 2009;0:gfp180v1-gfp180.
ABSTRACT | FULL TEXT  

Impact of a Multifaceted Intervention on Cholesterol Management in Primary Care Practices: Guideline Adherence for Heart Health Randomized Trial
Bertoni et al.
Arch Intern Med 2009;169:678-686.
ABSTRACT | FULL TEXT  

An evaluation by teachers of a decision aid for viewing bitewing radiographs
Mileman and van den Hout
Dentomaxillofac Radiol 2008;37:425-432.
ABSTRACT | FULL TEXT  

Adoption of Anesthesia Information Management Systems by Academic Departments in the United States
Egger Halbeis et al.
Anesth. Analg. 2008;107:1323-1329.
ABSTRACT | FULL TEXT  

Accuracy in the recording of pressure ulcers and prevention after implementing an electronic health record in hospital care
Gunningberg et al.
Qual Saf Health Care 2008;17:281-285.
ABSTRACT | FULL TEXT  

Self Determination Theory and Preventive Care Delivery: A Research Involving Outpatient Settings Network (RIOS Net) Study
Sussman et al.
J Am Board Fam Med 2008;21:282-292.
ABSTRACT | FULL TEXT  

Implementation of Evidence-Based Preventive Services Delivery Processes in Primary Care: An Oklahoma Physicians Resource/Research Network (OKPRN) Study
Mold et al.
J Am Board Fam Med 2008;21:334-344.
ABSTRACT | FULL TEXT  

A Randomized Trial of Electronic Clinical Reminders to Improve Medication Laboratory Monitoring
Matheny et al.
J. Am. Med. Inform. Assoc. 2008;15:424-429.
ABSTRACT | FULL TEXT  

What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians
Varonen et al.
Fam Pract 2008;25:162-167.
ABSTRACT | FULL TEXT  

A comparison of electronic records to paper records in mental health centers
Tsai and Bond
Int J Qual Health Care 2008;20:136-143.
ABSTRACT | FULL TEXT  

The Influence of a Physician's Use of a Diagnostic Decision Aid on the Malpractice Verdicts of Mock Jurors
Arkes et al.
Med Decis Making 2008;28:201-208.
ABSTRACT  

Randomized Trial of a Clinical Decision Support System: Impact on the Management of Children with Fever without Apparent Source
Roukema et al.
J. Am. Med. Inform. Assoc. 2008;15:107-113.
ABSTRACT | FULL TEXT  

Self-Reported Performance Improvement Strategies of Highly Successful Veterans Health Administration Facilities
Craig et al.
American Journal of Medical Quality 2007;22:438-444.
ABSTRACT  

Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses
Saleem et al.
J. Am. Med. Inform. Assoc. 2007;14:632-640.
ABSTRACT | FULL TEXT  

Adverse drug events in the elderly
Cresswell et al.
Br Med Bull 2007;83:259-274.
ABSTRACT | FULL TEXT  

Electronic Health Record Use and the Quality of Ambulatory Care in the United States
Linder et al.
Arch Intern Med 2007;167:1400-1405.
ABSTRACT | FULL TEXT  

Adoption of Order Entry with Decision Support for Chronic Care by Physician Organizations
Simon et al.
J. Am. Med. Inform. Assoc. 2007;14:432-439.
ABSTRACT | FULL TEXT  

A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network
Wright et al.
J. Am. Med. Inform. Assoc. 2007;14:489-496.
ABSTRACT | FULL TEXT  

Families' Health-Related Social Problems and Missed Referral Opportunities
Fleegler et al.
Pediatrics 2007;119:e1332-e1341.
ABSTRACT | FULL TEXT  

Pain Management in Hospitalized Cancer Patients: A Systematic Review
Goldberg and Morrison
JCO 2007;25:1792-1801.
ABSTRACT | FULL TEXT  

Patients Derogate Physicians Who Use a Computer-Assisted Diagnostic Aid
Arkes et al.
Med Decis Making 2007;27:189-202.
ABSTRACT  

The evolving science of translating research evidence into clinical practice
Scott
Evid. Based Med. 2007;12:4-7.
FULL TEXT  

Do Integrated Medical Groups Provide Higher-Quality Medical Care than Individual Practice Associations?
Mehrotra et al.
ANN INTERN MED 2006;145:826-833.
ABSTRACT | FULL TEXT  

Computer-based decision support for pediatric asthma management: description and feasibility of the Stop Asthma Clinical System
Shegog et al.
Health Informatics Journal 2006;12:259-273.
ABSTRACT  

Can computers improve patient care by primary health care workers in India?
Peters et al.
Int J Qual Health Care 2006;18:437-445.
ABSTRACT | FULL TEXT  

Use of a Personal Digital Assistant for Managing Antibiotic Prescribing for Outpatient Respiratory Tract Infections in Rural Communities
Rubin et al.
J. Am. Med. Inform. Assoc. 2006;13:627-634.
ABSTRACT | FULL TEXT  

Citation classics in patient safety research: an invitation to contribute to an online bibliography.
Lilford et al.
Qual Saf Health Care 2006;15:311-313.
ABSTRACT | FULL TEXT  

Prescribers' Responses to Alerts During Medication Ordering in the Long Term Care Setting
Judge et al.
J. Am. Med. Inform. Assoc. 2006;13:385-390.
ABSTRACT | FULL TEXT  

Biomedical Informatics Applications for Asthma Care: A Systematic Review
Sanders and Aronsky
J. Am. Med. Inform. Assoc. 2006;13:418-427.
ABSTRACT | FULL TEXT  

Getting physicians to accept new information technology: insights from case studies.
Lapointe and Rivard
CMAJ 2006;174:1573-1578.
ABSTRACT | FULL TEXT  

Information technology for optimizing the management of infectious diseases
Drew et al.
Am J Health Syst Pharm 2006;63:957-965.
FULL TEXT  

The Logic Behind a Multimethod Intervention to Improve Adherence to Clinical Practice Guidelines in a Nationwide Network of Primary Care Practices
Feifer et al.
Eval Health Prof 2006;29:65-88.
ABSTRACT  

A Practice-Sponsored Web Site to Help Patients Pursue Healthy Behaviors: An ACORN Study
Woolf et al.
Ann Fam Med 2006;4:148-152.
ABSTRACT | FULL TEXT  

Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs et al.
J. Am. Med. Inform. Assoc. 2006;13:138-147.
ABSTRACT | FULL TEXT  

The Potential of Training to Increase Acceptance and Use of Computerized Decision Support Systems for Medical Diagnosis
Lai et al.
Human Factors: The Journal of the Human Factors and Ergonomics Society 2006;48:95-108.
ABSTRACT  

Physician Evaluation after Medical Errors: Does Having a Computer Decision Aid Help or Hurt in Hindsight?
Pezzo and Pezzo
Med Decis Making 2006;26:48-56.
ABSTRACT  

Cross-sectional Analysis of Guidelines on Cardiovascular Disease Risk Factors: Going to Meet the Inconsistencies
van Wyk et al.
Med Decis Making 2006;26:57-62.
ABSTRACT  

Clinical Guidelines and Policies: Can they Improve Emergency Department Pain Management?
Ducharme
J Law Med Ethics 2005;33:783-790.
 

Information Technology as a Tool to Improve the Quality of American Indian Health Care
Sequist et al.
Am. J. Public Health 2005;95:2173-2179.
ABSTRACT | FULL TEXT  

A Randomized Outpatient Trial of a Decision-Support Information Technology Tool
Apkon et al.
Arch Intern Med 2005;165:2388-2394.
ABSTRACT | FULL TEXT  

Clinical Decision Support and Appropriateness of Antimicrobial Prescribing: A Randomized Trial
Samore et al.
JAMA 2005;294:2305-2314.
ABSTRACT | FULL TEXT  

Referral or follow-up?
Sullivan and Wyatt
BMJ 2005;331:1072-1074.
FULL TEXT  

Translation of Research Into Practice: Why We Can't "Just Do It"
Green and Seifert
J Am Board Fam Med 2005;18:541-545.
ABSTRACT | FULL TEXT  

Assessment of Screening, Treatment, and Prevention of Perinatal Infections in the Philadelphia Birth Cohort
Jessop et al.
American Journal of Medical Quality 2005;20:253-261.
ABSTRACT  

Safe chemotherapy administration: Using failure mode and effects analysis in computerized prescriber order entry
Kozakiewicz et al.
Am J Health Syst Pharm 2005;62:1813-1816.
FULL TEXT  

The Effect of Computerized Provider Order Entry on Medical Student Clerkship Experiences
Knight et al.
J. Am. Med. Inform. Assoc. 2005;12:554-560.
ABSTRACT | FULL TEXT  

Interventions to Regulate Ordering of Serum Magnesium Levels: Report of an Unintended Consequence of Decision Support
Rosenbloom et al.
J. Am. Med. Inform. Assoc. 2005;12:546-553.
ABSTRACT | FULL TEXT  

The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review
Poissant et al.
J. Am. Med. Inform. Assoc. 2005;12:505-516.
ABSTRACT | FULL TEXT  

Improving evidence based cardiac care and policy implementation over the patient journey: the potential of coronary heart disease registers
Clark et al.
Heart 2005;91:1127-1130.
ABSTRACT | FULL TEXT  

Diagnostic Error in Internal Medicine
Graber et al.
Arch Intern Med 2005;165:1493-1499.
ABSTRACT | FULL TEXT  

Impact of an Electronic Medical Record on Diabetes Quality of Care
O'Connor et al.
Ann Fam Med 2005;3:300-306.
ABSTRACT | FULL TEXT  

Handheld Computer-based Decision Support Reduces Patient Length of Stay and Antibiotic Prescribing in Critical Care
Sintchenko et al.
J. Am. Med. Inform. Assoc. 2005;12:398-402.
ABSTRACT | FULL TEXT  

A Randomized Trial of Electronic Clinical Reminders to Improve Quality of Care for Diabetes and Coronary Artery Disease
Sequist et al.
J. Am. Med. Inform. Assoc. 2005;12:431-437.
ABSTRACT | FULL TEXT  

Exploring Barriers and Facilitators to the Use of Computerized Clinical Reminders
Saleem et al.
J. Am. Med. Inform. Assoc. 2005;12:438-447.
ABSTRACT | FULL TEXT  

Health Information Technology and Physician-Patient Interactions: Impact of Computers on Communication during Outpatient Primary Care Visits
Hsu et al.
J. Am. Med. Inform. Assoc. 2005;12:474-480.
ABSTRACT | FULL TEXT  

Effect of CPOE User Interface Design on User-Initiated Access to Educational and Patient Information during Clinical Care
Rosenbloom et al.
J. Am. Med. Inform. Assoc. 2005;12:458-473.
ABSTRACT | FULL TEXT  

Challenges in Systematic Reviews: Synthesis of Topics Related to the Delivery, Organization, and Financing of Health Care
Bravata et al.
ANN INTERN MED 2005;142:1056-1065.
ABSTRACT | FULL TEXT  

Improving the ICU: Part 2
Garland
Chest 2005;127:2165-2179.
ABSTRACT | FULL TEXT  

Validity of ICD-9-CM Administrative Data for Determining Eligibility for Pneumococcal Vaccination Triggers
Coffin et al.
American Journal of Medical Quality 2005;20:158-163.
ABSTRACT  

Cluster Randomized Trials of Professional and Organizational Behavior Change Interventions in Health Care Settings
Grimshaw et al.
The ANNALS of the American Academy of Political and Social Science 2005;599:71-93.
ABSTRACT  

Identifying Individuals with a Reduced GFR Using Ambulatory Laboratory Database Surveillance
Garg et al.
J. Am. Soc. Nephrol. 2005;16:1433-1439.
ABSTRACT | FULL TEXT  

Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success
Kawamoto et al.
BMJ 2005;330:765.
ABSTRACT | FULL TEXT  

Electronic Alerts to Prevent Venous Thromboembolism among Hospitalized Patients
Kucher et al.
NEJM 2005;352:969-977.
ABSTRACT | FULL TEXT  

Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review
Garg et al.
JAMA 2005;293:1223-1238.
ABSTRACT | FULL TEXT  

Reducing Excessive Medication Administration in Hospitalized Adults With Renal Dysfunction
Nash et al.
American Journal of Medical Quality 2005;20:64-69.
ABSTRACT  

Opening the Black Box: Cognitive Strategies in Family Practice
Christensen et al.
Ann Fam Med 2005;3:144-150.
ABSTRACT | FULL TEXT  

Use of a Computerized Guideline for Glucose Regulation in the Intensive Care Unit Improved Both Guideline Adherence and Glucose Regulation
Rood et al.
J. Am. Med. Inform. Assoc. 2005;12:172-180.
ABSTRACT | FULL TEXT  

Quality of Diabetes Care in U.S. Academic Medical Centers: Low rates of medical regimen change
Grant et al.
Diabetes Care 2005;28:337-442.
ABSTRACT | FULL TEXT  

Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence
Haynes
Evid. Based Nurs. 2005;8:4-6.
FULL TEXT  

Should dentistry be part of the National Health Information Infrastructure?
SCHLEYER
Journal of the American Dental Association 2004;135:1687-1695.
ABSTRACT | FULL TEXT  

Access Denied; Care Impaired: The Benefits of Having Online Medical Information Available at the Point-of-Care
Finegan
Anesth. Analg. 2004;99:1450-1452.
ABSTRACT | FULL TEXT  

A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?
Delpierre et al.
Int J Qual Health Care 2004;16:407-416.
ABSTRACT | FULL TEXT  

Influence of guideline adherence on outcome in a randomised controlled trial on the efficacy of methotrexate with folate supplementation in rheumatoid arthritis
Fransen et al.
Ann Rheum Dis 2004;63:1222-1226.
ABSTRACT | FULL TEXT  

Impact of an Evidence-Based Computerized Decision Support System on Primary Care Prescription Costs
McMullin et al.
Ann Fam Med 2004;2:494-498.
ABSTRACT | FULL TEXT  

A Network-Based System to Improve Care for Schizophrenia: The Medical Informatics Network Tool (MINT)
Young et al.
J. Am. Med. Inform. Assoc. 2004;11:358-367.
ABSTRACT | FULL TEXT  

A Computerized Clinical Decision Support System as a Means of Implementing Depression Guidelines
Trivedi et al.
Psychiatr. Serv. 2004;55:879-885.
ABSTRACT | FULL TEXT  

From knowledge to practice in chronic cardiovascular disease: a long and winding road
Majumdar et al.
J Am Coll Cardiol 2004;43:1738-1742.
ABSTRACT | FULL TEXT  

The Epidemiology of Prescribing Errors: The Potential Impact of Computerized Prescriber Order Entry
Bobb et al.
Arch Intern Med 2004;164:785-792.
ABSTRACT | FULL TEXT  

Influencing Social Workers to Use Research Evidence in Practice: Lessons from Medicine and the Allied Health Professions
Gira et al.
Research on Social Work Practice 2004;14:68-79.
ABSTRACT  

A Conceptual Framework for Evaluating Information Technologies and Decision Support Systems for Bioterrorism Preparedness and Response
Bravata et al.
Med Decis Making 2004;24:192-206.
ABSTRACT  

Physicians' Use Of Electronic Medical Records: Barriers And Solutions
Miller and Sim
Health Aff (Millwood) 2004;23:116-126.
ABSTRACT | FULL TEXT  

Using evidence-based dissemination and implementation strategies to improve routine communication between general practitioners and community mental health teams
White and Marriott
Psychiatr. Bull. 2004;28:8-11.
ABSTRACT | FULL TEXT  

Comparative Impact of Guidelines, Clinical Data, and Decision Support on Prescribing Decisions: An Interactive Web Experiment with Simulated Cases
Sintchenko et al.
J. Am. Med. Inform. Assoc. 2004;11:71-77.
ABSTRACT | FULL TEXT  

Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality
Bates et al.
J. Am. Med. Inform. Assoc. 2003;10:523-530.
ABSTRACT | FULL TEXT  

Measuring the Impact of Diagnostic Decision Support on the Quality of Clinical Decision Making: Development of a Reliable and Valid Composite Score
Ramnarayan et al.
J. Am. Med. Inform. Assoc. 2003;10:563-572.
ABSTRACT | FULL TEXT  

The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care
Tamblyn et al.
CMAJ 2003;169:549-556.
ABSTRACT | FULL TEXT  

The Impact of Computerized Clinical Reminders on Physician Prescribing Behavior: Evidence From Community Oncology Practice
Kralj et al.
American Journal of Medical Quality 2003;18:197-203.
ABSTRACT  

AsthmaCritic: Issues in Designing a Noninquisitive Critiquing System for Daily Practice
Kuilboer et al.
J. Am. Med. Inform. Assoc. 2003;10:419-424.
ABSTRACT | FULL TEXT  

Medication misuse in hospitalized patients with renal impairment
SALOMON et al.
Int J Qual Health Care 2003;15:331-335.
ABSTRACT | FULL TEXT  

Impact of Population Management With Direct Physician Feedback on Care of Patients With Type 2 Diabetes
Grant et al.
Diabetes Care 2003;26:2275-2280.
ABSTRACT | FULL TEXT  

Electronic Technology: A Spark to Revitalize Primary Care?
Bodenheimer and Grumbach
JAMA 2003;290:259-264.
ABSTRACT | FULL TEXT  

Providers' Reactions to an Automated Health Maintenance Reminder System Incorporated into the Patient's Electronic Medical Record
Schellhase et al.
J Am Board Fam Med 2003;16:312-317.
ABSTRACT | FULL TEXT  

Evaluation of a Cardiac Diagnostic Program in a Typical Clinical Setting
Fraser et al.
J. Am. Med. Inform. Assoc. 2003;10:373-381.
ABSTRACT | FULL TEXT  

Effectiveness of a measurement feedback system on outcome in rheumatoid arthritis: a controlled clinical trial
Fransen et al.
Ann Rheum Dis 2003;62:624-629.
ABSTRACT | FULL TEXT  

Improving the Quality of Care -- Can We Practice What We Preach?
Steinberg
NEJM 2003;348:2681-2683.
FULL TEXT  

Effects of Computerized Physician Order Entry and Clinical Decision Support Systems on Medication Safety: A Systematic Review
Kaushal et al.
Arch Intern Med 2003;163:1409-1416.
ABSTRACT | FULL TEXT  

Improving Safety with Information Technology
Bates and Gawande
NEJM 2003;348:2526-2534.
FULL TEXT