This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (180)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Quality of Care, Other  • Review  • Prognosis/ Outcomes  • Alert me on articles by topic

A Systematic Review

Amit X. Garg, MD; Neill K. J. Adhikari, MD; Heather McDonald, MSc; M. Patricia Rosas-Arellano, MD, PhD; P. J. Devereaux, MD; Joseph Beyene, PhD; Justina Sam, BHSc; R. Brian Haynes, MD, PhD

JAMA. 2005;293:1223-1238.

Context  Developers of health care software have attributed improvements in patient care to these applications. As with any health care intervention, such claims require confirmation in clinical trials.

Objectives  To review controlled trials assessing the effects of computerized clinical decision support systems (CDSSs) and to identify study characteristics predicting benefit.

Data Sources  We updated our earlier reviews by searching the MEDLINE, EMBASE, Cochrane Library, Inspec, and ISI databases and consulting reference lists through September 2004. Authors of 64 primary studies confirmed data or provided additional information.

Study Selection  We included randomized and nonrandomized controlled trials that evaluated the effect of a CDSS compared with care provided without a CDSS on practitioner performance or patient outcomes.

Data Extraction  Teams of 2 reviewers independently abstracted data on methods, setting, CDSS and patient characteristics, and outcomes.

Data Synthesis  One hundred studies met our inclusion criteria. The number and methodologic quality of studies improved over time. The CDSS improved practitioner performance in 62 (64%) of the 97 studies assessing this outcome, including 4 (40%) of 10 diagnostic systems, 16 (76%) of 21 reminder systems, 23 (62%) of 37 disease management systems, and 19 (66%) of 29 drug-dosing or prescribing systems. Fifty-two trials assessed 1 or more patient outcomes, of which 7 trials (13%) reported improvements. Improved practitioner performance was associated with CDSSs that automatically prompted users compared with requiring users to activate the system (success in 73% of trials vs 47%; P = .02) and studies in which the authors also developed the CDSS software compared with studies in which the authors were not the developers (74% success vs 28%; respectively, P = .001).

Conclusions  Many CDSSs improve practitioner performance. To date, the effects on patient outcomes remain understudied and, when studied, inconsistent.

Author Affiliations: Division of Nephrology (Drs Garg and Rosas-Arellano) and Department of Epidemiology and Biostatistics (Dr Garg), University of Western Ontario, London; Departments of Clinical Epidemiology and Biostatistics (Drs Garg, Adhikari, Devereaux, and Haynes and Ms McDonald) and Medicine (Drs Devereaux and Haynes), McMaster University, Hamilton, Ontario; Department of Critical Care Medicine, Sunnybrook and Women’s College Health Sciences Centre and Interdepartmental Division of Critical Care (Dr Adhikari), Population Health Sciences, Hospital for Sick Children (Dr Beyene), and Faculty of Medicine (Ms Sam), University of Toronto, Toronto, Ontario.

RELATED ARTICLES

Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Ross Koppel, Joshua P. Metlay, Abigail Cohen, Brian Abaluck, A. Russell Localio, Stephen E. Kimmel, and Brian L. Strom
JAMA. 2005;293(10):1197-1203.
ABSTRACT | FULL TEXT  

Computer Technology and Clinical Work: Still Waiting for Godot
Robert L. Wears and Marc Berg
JAMA. 2005;293(10):1261-1263.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Methodologic Issues in Health Informatics Trials: The Complexities of Complex Interventions
Shcherbatykh et al.
J. Am. Med. Inform. Assoc. 2008;15:575-580.
ABSTRACT | FULL TEXT  

Ordering Molecular Genetic Tests and Reporting Results: Practices in Laboratory and Clinical Settings
Lubin et al.
J. Mol. Diagn. 2008;10:459-468.
ABSTRACT | FULL TEXT  

Knowledge-to-action cycle
Straus and Holroyd-Leduc
Evid. Based Med. 2008;13:98-100.
FULL TEXT  

"Smart Forms" in an Electronic Medical Record: Documentation-based Clinical Decision Support to Improve Disease Management
Schnipper et al.
J. Am. Med. Inform. Assoc. 2008;15:513-523.
ABSTRACT | FULL TEXT  

The Informatics Opportunities at the Intersection of Patient Safety and Clinical Informatics
Kilbridge and Classen
J. Am. Med. Inform. Assoc. 2008;15:397-407.
ABSTRACT | FULL TEXT  

Evaluation of the GIDEON Expert Computer Program for the Diagnosis of Imported Febrile Illnesses
Bottieau et al.
Med Decis Making 2008;28:435-442.
ABSTRACT  

What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians
Varonen et al.
Fam Pract 2008;25:162-167.
ABSTRACT | FULL TEXT  

Potential for improving patient safety by computerized decision support systems
Delaney
Fam Pract 2008;25:137-138.
FULL TEXT  

Improving prevention in primary care: Evaluating the sustainability of outreach facilitation
Hogg et al.
cfp 2008;54:712-720.
ABSTRACT | FULL TEXT  

Prompting Clinicians about Preventive Care Measures: A Systematic Review of Randomized Controlled Trials
Dexheimer et al.
J. Am. Med. Inform. Assoc. 2008;15:311-320.
ABSTRACT | FULL TEXT  

EHR Safety: The Way Forward to Safe and Effective Systems
Walker et al.
J. Am. Med. Inform. Assoc. 2008;15:272-277.
ABSTRACT | FULL TEXT  

Causes of preventable drug-related hospital admissions: a qualitative study
Howard et al.
Qual Saf Health Care 2008;17:109-116.
ABSTRACT | FULL TEXT  

The Influence of a Physician's Use of a Diagnostic Decision Aid on the Malpractice Verdicts of Mock Jurors
Arkes et al.
Med Decis Making 2008;28:201-208.
ABSTRACT  

A Randomized Effectiveness Trial of a Clinical Informatics Consult Service: Impact on Evidence-based Decision-making and Knowledge Implementation
Mulvaney et al.
J. Am. Med. Inform. Assoc. 2008;15:203-211.
ABSTRACT | FULL TEXT  

Contextual Implementation Model: A Framework for Assisting Clinical Information System Implementations
Callen et al.
J. Am. Med. Inform. Assoc. 2008;15:255-262.
ABSTRACT | FULL TEXT  

Using Computer-Based Decision Support to Close the "Know Do" Gap in Lipid-Lowering Therapy
Avorn and Choudhry
Circulation 2008;117:336-337.
FULL TEXT  

Electronic Alerts Versus On-Demand Decision Support to Improve Dyslipidemia Treatment: A Cluster Randomized Controlled Trial
van Wyk et al.
Circulation 2008;117:371-378.
ABSTRACT | FULL TEXT  

Impact of a spirometry expert system on general practitioners' decision making
Poels et al.
Eur Respir J 2008;31:84-92.
ABSTRACT | FULL TEXT  

Randomized Trial of a Clinical Decision Support System: Impact on the Management of Children with Fever without Apparent Source
Roukema et al.
J. Am. Med. Inform. Assoc. 2008;15:107-113.
ABSTRACT | FULL TEXT  

Self-Reported Performance Improvement Strategies of Highly Successful Veterans Health Administration Facilities
Craig et al.
American Journal of Medical Quality 2007;22:438-444.
ABSTRACT  

Patterns of Use of Handheld Clinical Decision Support Tools in the Clinical Setting
Yu et al.
Med Decis Making 2007;27:744-753.
ABSTRACT  

Prediction Is Difficult, Particularly About the Future
Jackson and Wells
Arch Intern Med 2007;167:2286-2287.
FULL TEXT  

Impact of Electronic Alerts on Isolation Precautions for Patients With Multidrug-Resistant Bacteria
Kac et al.
Arch Intern Med 2007;167:2086-2090.
ABSTRACT | FULL TEXT  

Effects of an integrated clinical information system on medication safety in a multi-hospital setting
Mahoney et al.
Am J Health Syst Pharm 2007;64:1969-1977.
ABSTRACT | FULL TEXT  

Tackling therapeutic inertia: role of treatment data in quality indicators
Guthrie et al.
BMJ 2007;335:542-544.
FULL TEXT  

Unintended Consequences of Information Technologies in Health Care An Interactive Sociotechnical Analysis
Harrison et al.
J. Am. Med. Inform. Assoc. 2007;14:542-549.
ABSTRACT | FULL TEXT  

Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses
Saleem et al.
J. Am. Med. Inform. Assoc. 2007;14:632-640.
ABSTRACT | FULL TEXT  

Strategic approach for improving the medication-use process in health systems: The high-performance pharmacy practice framework
Vermeulen et al.
Am J Health Syst Pharm 2007;64:1699-1710.
ABSTRACT | FULL TEXT  

Electronic Health Record Use and the Quality of Ambulatory Care in the United States
Linder et al.
Arch Intern Med 2007;167:1400-1405.
ABSTRACT | FULL TEXT  

A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network
Wright et al.
J. Am. Med. Inform. Assoc. 2007;14:489-496.
ABSTRACT | FULL TEXT  

The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry
Ash et al.
J. Am. Med. Inform. Assoc. 2007;14:415-423.
ABSTRACT | FULL TEXT  

Hyperglycemia management in the hospital setting
Hassan
Am J Health Syst Pharm 2007;64:S9-S14.
ABSTRACT | FULL TEXT  

Electronic Medical Records and Diabetes Quality of Care: Results From a Sample of Family Medicine Practices
Crosson et al.
Ann Fam Med 2007;5:209-215.
ABSTRACT | FULL TEXT  

Different Paths to High-Quality Care: Three Archetypes of Top-Performing Practice Sites
Feifer et al.
Ann Fam Med 2007;5:233-241.
ABSTRACT | FULL TEXT  

A Viewpoint on Evidence-based Health Informatics, Based on a Pilot Survey on Evaluation Studies in Health Care Informatics
Ammenwerth and de Keizer
J. Am. Med. Inform. Assoc. 2007;14:368-371.
ABSTRACT | FULL TEXT  

Informatics Systems to Promote Improved Care for Chronic Illness: A Literature Review
Dorr et al.
J. Am. Med. Inform. Assoc. 2007;14:156-163.
ABSTRACT | FULL TEXT  

Proposal for Fulfilling Strategic Objectives of the U.S. Roadmap for National Action on Decision Support through a Service-oriented Architecture Leveraging HL7 Services
Kawamoto and Lobach
J. Am. Med. Inform. Assoc. 2007;14:146-155.
ABSTRACT | FULL TEXT  

Advancing Evidence-Based Care For Diabetes: Lessons From The Veterans Health Administration
Kupersmith et al.
Health Aff (Millwood) 2007;26:w156-w168.
ABSTRACT | FULL TEXT  

Utility of Nerve Conduction Studies for Carpal Tunnel Syndrome by Family Medicine, Primary Care, and Internal Medicine Physicians
Megerian et al.
J Am Board Fam Med 2007;20:60-64.
ABSTRACT | FULL TEXT  

Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review
Kuperman et al.
J. Am. Med. Inform. Assoc. 2007;14:29-40.
ABSTRACT | FULL TEXT  

Evaluation and Certification of Computerized Provider Order Entry Systems
Classen et al.
J. Am. Med. Inform. Assoc. 2007;14:48-55.
ABSTRACT | FULL TEXT  

Electronic Medical Records and Their Impact on Resident and Medical Student Education
Keenan et al.
Acad. Psychiatry 2006;30:522-527.
ABSTRACT | FULL TEXT  

Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial
Paul et al.
J Antimicrob Chemother 2006;58:1238-1245.
ABSTRACT | FULL TEXT  

The Stroke-Thrombolytic Predictive Instrument: A Predictive Instrument for Intravenous Thrombolysis in Acute Ischemic Stroke
Kent et al.
Stroke 2006;37:2957-2962.
ABSTRACT | FULL TEXT  

The Stroke-Thrombolytic Predictive Instrument Provides Valid Quantitative Estimates of Outcome Probabilities and Aids Clinical Decision-Making
Demaerschalk
Stroke 2006;37:2865-2866.
FULL TEXT  

Use of a Personal Digital Assistant for Managing Antibiotic Prescribing for Outpatient Respiratory Tract Infections in Rural Communities
Rubin et al.
J. Am. Med. Inform. Assoc. 2006;13:627-634.
ABSTRACT | FULL TEXT  

Effectiveness of Clinician-selected Electronic Information Resources for Answering Primary Care Physicians' Information Needs
McKibbon and Fridsma
J. Am. Med. Inform. Assoc. 2006;13:653-659.
ABSTRACT | FULL TEXT  

The Chronic Kidney Disease Epidemic: Stepping Back and Looking Forward
Kiberd
J. Am. Soc. Nephrol. 2006;17:2967-2973.
ABSTRACT | FULL TEXT  

Development and Testing of a Scale to Assess Physician Attitudes about Handheld Computers with Decision Support
Ray et al.
J. Am. Med. Inform. Assoc. 2006;13:567-572.
ABSTRACT | FULL TEXT  

What Interventions Should Pharmacists Employ to Impact Health Practitioners' Prescribing Practices?
Grindrod et al.
The Annals of Pharmacotherapy 2006;40:1546-1557.
ABSTRACT | FULL TEXT  

Impact of a Computerized Clinical Decision Support System on Reducing Inappropriate Antimicrobial Use: A Randomized Controlled Trial
McGregor et al.
J. Am. Med. Inform. Assoc. 2006;13:378-384.
ABSTRACT | FULL TEXT  

Using Commercial Knowledge Bases for Clinical Decision Support: Opportunities, Hurdles, and Recommendations
Kuperman et al.
J. Am. Med. Inform. Assoc. 2006;13:369-371.
FULL TEXT  

The NHS programme for information technology
Keen
BMJ 2006;333:3-4.
FULL TEXT  

Getting physicians to accept new information technology: insights from case studies.
Lapointe and Rivard
CMAJ 2006;174:1573-1578.
ABSTRACT | FULL TEXT  

Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care
Chaudhry et al.
ANN INTERN MED 2006;144:742-752.
ABSTRACT | FULL TEXT  

Information technology for optimizing the management of infectious diseases
Drew et al.
Am J Health Syst Pharm 2006;63:957-965.
FULL TEXT  

Computerization Can Create Safety Hazards: A Bar-Coding Near Miss
McDonald
ANN INTERN MED 2006;144:510-516.
ABSTRACT | FULL TEXT  

Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems.
Kilbridge et al.
Qual Saf Health Care 2006;15:81-84.
ABSTRACT | FULL TEXT  

Improving Ambulatory Prescribing Safety with a Handheld Decision Support System: A Randomized Controlled Trial
Berner et al.
J. Am. Med. Inform. Assoc. 2006;13:171-179.
ABSTRACT | FULL TEXT  

Information Technology and Cancer Prevention
Jimbo et al.
CA Cancer J Clin 2006;56:26-36.
ABSTRACT | FULL TEXT  

These Are the Technologies That Try Men's Souls: Common-Sense Health Information Technology
Gesteland et al.
Pediatrics 2006;117:216-217.
FULL TEXT  

Creating a Safer Health Care System: Finding the Constraint
Pauker et al.
JAMA 2005;294:2906-2908.
FULL TEXT  

The Tension Between E-Health Innovation and E-Valuation
Garg and Tonelli
Arch Intern Med 2005;165:2329-2330.
FULL TEXT  

A Randomized Outpatient Trial of a Decision-Support Information Technology Tool
Apkon et al.
Arch Intern Med 2005;165:2388-2394.
ABSTRACT | FULL TEXT  

Informatics Challenges for the Impending Patient Information Explosion
Berner and Moss
J. Am. Med. Inform. Assoc. 2005;12:614-617.
ABSTRACT | FULL TEXT  

Antimicrobial Stewardship Programs in Health Care Systems
MacDougall and Polk
Clin. Microbiol. Rev. 2005;18:638-656.
ABSTRACT | FULL TEXT  

The Effect of Computerized Provider Order Entry on Medical Student Clerkship Experiences
Knight et al.
J. Am. Med. Inform. Assoc. 2005;12:554-560.
ABSTRACT | FULL TEXT  

Quality and Quality Improvement in Forensic Mental Health Evaluations
Wettstein
J Am Acad Psychiatry Law 2005;33:158-175.
ABSTRACT | FULL TEXT  

What makes a good clinical decision support system
Purcell
BMJ 2005;330:740-741.
FULL TEXT  

What's new in the other general journals
Tonks
BMJ 2005;330:623-624.
FULL TEXT  

Computer Technology and Clinical Work: Still Waiting for Godot
Wears and Berg
JAMA 2005;293:1261-1263.
FULL TEXT