This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (277)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Quality of Care  • Quality of Care, Other  • Review  • Prognosis/ Outcomes  • Alert me on articles by topic  Social Bookmarking   What's this?

A Systematic Review

Amit X. Garg, MD; Neill K. J. Adhikari, MD; Heather McDonald, MSc; M. Patricia Rosas-Arellano, MD, PhD; P. J. Devereaux, MD; Joseph Beyene, PhD; Justina Sam, BHSc; R. Brian Haynes, MD, PhD

JAMA. 2005;293:1223-1238.

Context  Developers of health care software have attributed improvements in patient care to these applications. As with any health care intervention, such claims require confirmation in clinical trials.

Objectives  To review controlled trials assessing the effects of computerized clinical decision support systems (CDSSs) and to identify study characteristics predicting benefit.

Data Sources  We updated our earlier reviews by searching the MEDLINE, EMBASE, Cochrane Library, Inspec, and ISI databases and consulting reference lists through September 2004. Authors of 64 primary studies confirmed data or provided additional information.

Study Selection  We included randomized and nonrandomized controlled trials that evaluated the effect of a CDSS compared with care provided without a CDSS on practitioner performance or patient outcomes.

Data Extraction  Teams of 2 reviewers independently abstracted data on methods, setting, CDSS and patient characteristics, and outcomes.

Data Synthesis  One hundred studies met our inclusion criteria. The number and methodologic quality of studies improved over time. The CDSS improved practitioner performance in 62 (64%) of the 97 studies assessing this outcome, including 4 (40%) of 10 diagnostic systems, 16 (76%) of 21 reminder systems, 23 (62%) of 37 disease management systems, and 19 (66%) of 29 drug-dosing or prescribing systems. Fifty-two trials assessed 1 or more patient outcomes, of which 7 trials (13%) reported improvements. Improved practitioner performance was associated with CDSSs that automatically prompted users compared with requiring users to activate the system (success in 73% of trials vs 47%; P = .02) and studies in which the authors also developed the CDSS software compared with studies in which the authors were not the developers (74% success vs 28%; respectively, P = .001).

Conclusions  Many CDSSs improve practitioner performance. To date, the effects on patient outcomes remain understudied and, when studied, inconsistent.

Author Affiliations: Division of Nephrology (Drs Garg and Rosas-Arellano) and Department of Epidemiology and Biostatistics (Dr Garg), University of Western Ontario, London; Departments of Clinical Epidemiology and Biostatistics (Drs Garg, Adhikari, Devereaux, and Haynes and Ms McDonald) and Medicine (Drs Devereaux and Haynes), McMaster University, Hamilton, Ontario; Department of Critical Care Medicine, Sunnybrook and Women’s College Health Sciences Centre and Interdepartmental Division of Critical Care (Dr Adhikari), Population Health Sciences, Hospital for Sick Children (Dr Beyene), and Faculty of Medicine (Ms Sam), University of Toronto, Toronto, Ontario.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

RELATED ARTICLES

Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Ross Koppel, Joshua P. Metlay, Abigail Cohen, Brian Abaluck, A. Russell Localio, Stephen E. Kimmel, and Brian L. Strom
JAMA. 2005;293(10):1197-1203.
ABSTRACT | FULL TEXT  

Computer Technology and Clinical Work: Still Waiting for Godot
Robert L. Wears and Marc Berg
JAMA. 2005;293(10):1261-1263.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

A Computerized Handheld Decision-Support System to Improve Pulmonary Embolism Diagnosis: A Randomized Trial
Roy et al.
ANN INTERN MED 2009;151:677-686.
ABSTRACT | FULL TEXT  

Use of pharmacy informatics resources by clinical pharmacy services in acute care hospitals
Matsuura and Weeks
Am J Health Syst Pharm 2009;66:1934-1938.
ABSTRACT | FULL TEXT  

From practice guidelines to clinical decision support: closing the loop
Fox et al.
JRSM 2009;102:464-473.
FULL TEXT  

Bayesian Information Fusion Networks for Biosurveillance Applications
Mnatsakanyan et al.
J. Am. Med. Inform. Assoc. 2009;16:855-863.
ABSTRACT | FULL TEXT  

Getting a better grip on research: the comfort of opinion
Maskrey et al.
InnovAiT 2009;2:679-686.
ABSTRACT | FULL TEXT  

Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model
DiCenso et al.
Evid. Based Nurs. 2009;12:99-101.
FULL TEXT  

Computerized Decision Support for the Cardiovascular Clinician: Applications for Venous Thromboembolism Prevention and Beyond
Piazza and Goldhaber
Circulation 2009;120:1133-1137.
FULL TEXT  

Clinical Decision Support Capabilities of Commercially-available Clinical Information Systems
Wright et al.
J Am Med Inform Assoc 2009;16:637-644.
ABSTRACT | FULL TEXT  

The Use of Wireless E-Mail to Improve Healthcare Team Communication
O'Connor et al.
J Am Med Inform Assoc 2009;16:705-713.
ABSTRACT | FULL TEXT  

The Use of Wireless E-Mail to Improve Healthcare Team Communication
O'Connor et al.
J. Am. Med. Inform. Assoc. 2009;16:705-713.
ABSTRACT | FULL TEXT  

Diagnosis of venous thromboembolism
Roy
BMJ 2009;339:b2799-b2799.
FULL TEXT  

Clinical prediction rules
Falk and Fahey
BMJ 2009;339:b2899-b2899.
FULL TEXT  

Evidence-based diagnosis in general practice: needs both robust evidence and sophisticated electronic health record systems
Delaney
Fam Pract 2009;26:239-240.
FULL TEXT  

Improving primary care for patients with complex chronic diseases: Can health information technology play a role?
Grant and Middleton
CMAJ 2009;181:17-18.
FULL TEXT  

Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial
Holbrook et al.
CMAJ 2009;181:37-44.
ABSTRACT | FULL TEXT  

The Relationship between Electronic Health Record Use and Quality of Care over Time
Zhou et al.
J Am Med Inform Assoc 2009;16:457-464.
ABSTRACT | FULL TEXT  

Physicians' Use of Key Functions in Electronic Health Records from 2005 to 2007: A Statewide Survey
Simon et al.
J Am Med Inform Assoc 2009;16:465-470.
ABSTRACT | FULL TEXT  

What Evidence Supports the Use of Computerized Alerts and Prompts to Improve Clinicians' Prescribing Behavior?
Schedlbauer et al.
J Am Med Inform Assoc 2009;16:531-538.
ABSTRACT | FULL TEXT  

What Evidence Supports the Use of Computerized Alerts and Prompts to Improve Clinicians' Prescribing Behavior?
Schedlbauer et al.
J. Am. Med. Inform. Assoc. 2009;16:531-538.
ABSTRACT | FULL TEXT  

Physicians' Use of Key Functions in Electronic Health Records from 2005 to 2007: A Statewide Survey
Simon et al.
J. Am. Med. Inform. Assoc. 2009;16:465-470.
ABSTRACT | FULL TEXT  

Ambulatory Quality Improvement in Academic Medical Centers: A Changing Landscape
Leas et al.
American Journal of Medical Quality 2009;24:287-294.
ABSTRACT  

Implementing Evidence-Based Practices for People With Schizophrenia
Drake et al.
Schizophr Bull 2009;35:704-713.
ABSTRACT | FULL TEXT  

Long-term effects of a multifaceted intervention to encourage the choice of the oral route for proton pump inhibitors: an interrupted time-series analysis
Colombet et al.
Qual Saf Health Care 2009;18:232-235.
ABSTRACT | FULL TEXT  

Implementation of the Enteral Nutrition Practice Recommendations
Brantley
Nutr Clin Pract 2009;24:335-343.
ABSTRACT | FULL TEXT  

Medication errors in critical care: risk factors, prevention and disclosure
Camire et al.
CMAJ 2009;180:936-943.
FULL TEXT  

Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation
Goud et al.
BMJ 2009;338:b1440-b1440.
ABSTRACT | FULL TEXT  

The emerging role of PDAs in information use and clinical decision making
Doran
Evid. Based Nurs. 2009;12:35-38.
FULL TEXT  

The Impact of an Electronic Reminder on the Use of Alarms After Separation from Cardiopulmonary Bypass
Eden et al.
Anesth. Analg. 2009;108:1203-1208.
ABSTRACT | FULL TEXT  

Diagnostic Errors--The Next Frontier for Patient Safety
Newman-Toker and Pronovost
JAMA 2009;301:1060-1062.
FULL TEXT  

Effect of Alerts for Drug Dosage Adjustment in Inpatients with Renal Insufficiency
Sellier et al.
J. Am. Med. Inform. Assoc. 2009;16:203-210.
ABSTRACT | FULL TEXT  

Health Information Technology And Patient Safety: Evidence From Panel Data
Parente and McCullough
Health Aff (Millwood) 2009;28:357-360.
ABSTRACT | FULL TEXT  

From Tasks To Processes: The Case For Changing Health Information Technology To Improve Health Care
Walker and Carayon
Health Aff (Millwood) 2009;28:467-477.
ABSTRACT | FULL TEXT  

Clinician Perspectives about Molecular Genetic Testing for Heritable Conditions and Development of a Clinician-Friendly Laboratory Report
Lubin et al.
J. Mol. Diagn. 2009;11:162-171.
ABSTRACT | FULL TEXT  

Testing our understanding of tests
Phillips and Westwood
Arch. Dis. Child. 2009;94:178-179.
FULL TEXT  

Systematic Review of Interventions to Improve Prescribing
Ostini et al.
The Annals of Pharmacotherapy 2009;43:502-513.
ABSTRACT | FULL TEXT  

Stroke prevention in atrial fibrillation: better use of anticoagulation and new agents will lead to improved outcomes
Nieuwlaat and Connolly
Heart 2009;95:95-97.
FULL TEXT  

Increasing Reliability of APACHE II Scores in a Medical-Surgical Intensive Care Unit: A Quality Improvement Study
Donahoe et al.
Am J Crit Care 2009;18:58-64.
ABSTRACT | FULL TEXT  

Use of an Electronic Medical Record System to Support Primary Care Recommendations to Prevent, Identify, and Manage Childhood Obesity
Rattay et al.
Pediatrics 2009;123:S100-S107.
ABSTRACT | FULL TEXT  

Effects of algorithm for diagnosis of active labour: cluster randomised trial
Cheyne et al.
BMJ 2008;337:a2396-a2396.
ABSTRACT | FULL TEXT  

Combined Task Delegation, Computerized Decision Support, and Feedback Improve Cardiovascular Risk for Type 2 Diabetic Patients: A cluster randomized trial in primary care
Cleveringa et al.
Diabetes Care 2008;31:2273-2275.
ABSTRACT | FULL TEXT  

Electronic Health Records and Malpractice Claims in Office Practice
Virapongse et al.
Arch Intern Med 2008;168:2362-2367.
ABSTRACT | FULL TEXT  

Benefit of Oral Anticoagulant Over Antiplatelet Therapy in Atrial Fibrillation Depends on the Quality of International Normalized Ratio Control Achieved by Centers and Countries as Measured by Time in Therapeutic Range
Connolly et al.
Circulation 2008;118:2029-2037.
ABSTRACT | FULL TEXT  

Improving Care, Improving Performance, or Just Improving Numbers?
Luchins
Psychiatr. Serv. 2008;59:1328-1330.
ABSTRACT | FULL TEXT  

Prevention, Diagnosis, and Management of Osteoporosis-Related Fracture: A Multifactoral Osteopathic Approach
Gronholz
JAOA: Journal of the American Osteopathic Association 2008;108:575-585.
ABSTRACT | FULL TEXT  

Practice-Linked Online Personal Health Records for Type 2 Diabetes Mellitus: A Randomized Controlled Trial
Grant et al.
Arch Intern Med 2008;168:1776-1782.
ABSTRACT | FULL TEXT  

Methodologic Issues in Health Informatics Trials: The Complexities of Complex Interventions
Shcherbatykh et al.
J. Am. Med. Inform. Assoc. 2008;15:575-580.
ABSTRACT | FULL TEXT  

Ordering Molecular Genetic Tests and Reporting Results: Practices in Laboratory and Clinical Settings
Lubin et al.
J. Mol. Diagn. 2008;10:459-468.
ABSTRACT | FULL TEXT  

Knowledge-to-action cycle
Straus and Holroyd-Leduc
Evid. Based Med. 2008;13:98-100.
FULL TEXT  

"Smart Forms" in an Electronic Medical Record: Documentation-based Clinical Decision Support to Improve Disease Management
Schnipper et al.
J. Am. Med. Inform. Assoc. 2008;15:513-523.
ABSTRACT | FULL TEXT  

The Informatics Opportunities at the Intersection of Patient Safety and Clinical Informatics
Kilbridge and Classen
J. Am. Med. Inform. Assoc. 2008;15:397-407.
ABSTRACT | FULL TEXT  

Next Generation of Health Information Tools: Where Do We Go From Here?
Grant
Mayo Clin Proc. 2008;83:745-746.
FULL TEXT  

Situation Awareness: Review of Mica Endsley's 1995 Articles on Situation Awareness Theory and Measurement
Wickens
Human Factors: The Journal of the Human Factors and Ergonomics Society 2008;50:397-403.
ABSTRACT  

Humans: Still Vital After All These Years of Automation
Parasuraman and Wickens
Human Factors: The Journal of the Human Factors and Ergonomics Society 2008;50:511-520.
ABSTRACT  

Evaluation of the GIDEON Expert Computer Program for the Diagnosis of Imported Febrile Illnesses
Bottieau et al.
Med Decis Making 2008;28:435-442.
ABSTRACT  

What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians
Varonen et al.
Fam Pract 2008;25:162-167.
ABSTRACT | FULL TEXT  

Potential for improving patient safety by computerized decision support systems
Delaney
Fam Pract 2008;25:137-138.
FULL TEXT  

Improving prevention in primary care: Evaluating the sustainability of outreach facilitation
Hogg et al.
cfp 2008;54:712-720.
ABSTRACT | FULL TEXT  

Prompting Clinicians about Preventive Care Measures: A Systematic Review of Randomized Controlled Trials
Dexheimer et al.
J. Am. Med. Inform. Assoc. 2008;15:311-320.
ABSTRACT | FULL TEXT  

EHR Safety: The Way Forward to Safe and Effective Systems
Walker et al.
J. Am. Med. Inform. Assoc. 2008;15:272-277.
ABSTRACT | FULL TEXT  

Causes of preventable drug-related hospital admissions: a qualitative study
Howard et al.
Qual Saf Health Care 2008;17:109-116.
ABSTRACT | FULL TEXT  

A Randomized Effectiveness Trial of a Clinical Informatics Consult Service: Impact on Evidence-based Decision-making and Knowledge Implementation
Mulvaney et al.
J. Am. Med. Inform. Assoc. 2008;15:203-211.
ABSTRACT | FULL TEXT  

Contextual Implementation Model: A Framework for Assisting Clinical Information System Implementations
Callen et al.
J. Am. Med. Inform. Assoc. 2008;15:255-262.
ABSTRACT | FULL TEXT  

The Influence of a Physician's Use of a Diagnostic Decision Aid on the Malpractice Verdicts of Mock Jurors
Arkes et al.
Med Decis Making 2008;28:201-208.
ABSTRACT  

Using Computer-Based Decision Support to Close the "Know Do" Gap in Lipid-Lowering Therapy
Avorn and Choudhry
Circulation 2008;117:336-337.
FULL TEXT  

Electronic Alerts Versus On-Demand Decision Support to Improve Dyslipidemia Treatment: A Cluster Randomized Controlled Trial
van Wyk et al.
Circulation 2008;117:371-378.
ABSTRACT | FULL TEXT  

Impact of a spirometry expert system on general practitioners' decision making
Poels et al.
Eur Respir J 2008;31:84-92.
ABSTRACT | FULL TEXT  

Randomized Trial of a Clinical Decision Support System: Impact on the Management of Children with Fever without Apparent Source
Roukema et al.
J. Am. Med. Inform. Assoc. 2008;15:107-113.
ABSTRACT | FULL TEXT  

Self-Reported Performance Improvement Strategies of Highly Successful Veterans Health Administration Facilities
Craig et al.
American Journal of Medical Quality 2007;22:438-444.
ABSTRACT  

Patterns of Use of Handheld Clinical Decision Support Tools in the Clinical Setting
Yu et al.
Med Decis Making 2007;27:744-753.
ABSTRACT  

Prediction Is Difficult, Particularly About the Future
Jackson and Wells
Arch Intern Med 2007;167:2286-2287.
FULL TEXT  

Impact of Electronic Alerts on Isolation Precautions for Patients With Multidrug-Resistant Bacteria
Kac et al.
Arch Intern Med 2007;167:2086-2090.
ABSTRACT | FULL TEXT  

Effects of an integrated clinical information system on medication safety in a multi-hospital setting
Mahoney et al.
Am J Health Syst Pharm 2007;64:1969-1977.
ABSTRACT | FULL TEXT  

Tackling therapeutic inertia: role of treatment data in quality indicators
Guthrie et al.
BMJ 2007;335:542-544.
FULL TEXT  

Unintended Consequences of Information Technologies in Health Care An Interactive Sociotechnical Analysis
Harrison et al.
J. Am. Med. Inform. Assoc. 2007;14:542-549.
ABSTRACT | FULL TEXT  

Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses
Saleem et al.
J. Am. Med. Inform. Assoc. 2007;14:632-640.
ABSTRACT | FULL TEXT  

Strategic approach for improving the medication-use process in health systems: The high-performance pharmacy practice framework
Vermeulen et al.
Am J Health Syst Pharm 2007;64:1699-1710.
ABSTRACT | FULL TEXT  

Electronic Health Record Use and the Quality of Ambulatory Care in the United States
Linder et al.
Arch Intern Med 2007;167:1400-1405.
ABSTRACT | FULL TEXT  

A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network
Wright et al.
J. Am. Med. Inform. Assoc. 2007;14:489-496.
ABSTRACT | FULL TEXT  

The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry
Ash et al.
J. Am. Med. Inform. Assoc. 2007;14:415-423.
ABSTRACT | FULL TEXT  

Hyperglycemia management in the hospital setting
Hassan
Am J Health Syst Pharm 2007;64:S9-S14.
ABSTRACT | FULL TEXT  

Electronic Medical Records and Diabetes Quality of Care: Results From a Sample of Family Medicine Practices
Crosson et al.
Ann Fam Med 2007;5:209-215.
ABSTRACT | FULL TEXT  

Different Paths to High-Quality Care: Three Archetypes of Top-Performing Practice Sites
Feifer et al.
Ann Fam Med 2007;5:233-241.
ABSTRACT | FULL TEXT  

A Viewpoint on Evidence-based Health Informatics, Based on a Pilot Survey on Evaluation Studies in Health Care Informatics
Ammenwerth and de Keizer
J. Am. Med. Inform. Assoc. 2007;14:368-371.
ABSTRACT | FULL TEXT  

Informatics Systems to Promote Improved Care for Chronic Illness: A Literature Review
Dorr et al.
J. Am. Med. Inform. Assoc. 2007;14:156-163.
ABSTRACT | FULL TEXT  

Proposal for Fulfilling Strategic Objectives of the U.S. Roadmap for National Action on Decision Support through a Service-oriented Architecture Leveraging HL7 Services
Kawamoto and Lobach
J. Am. Med. Inform. Assoc. 2007;14:146-155.
ABSTRACT | FULL TEXT  

Advancing Evidence-Based Care For Diabetes: Lessons From The Veterans Health Administration
Kupersmith et al.
Health Aff (Millwood) 2007;26:w156-w168.
ABSTRACT | FULL TEXT  

Utility of Nerve Conduction Studies for Carpal Tunnel Syndrome by Family Medicine, Primary Care, and Internal Medicine Physicians
Megerian et al.
J Am Board Fam Med 2007;20:60-64.
ABSTRACT | FULL TEXT  

Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review
Kuperman et al.
J. Am. Med. Inform. Assoc. 2007;14:29-40.
ABSTRACT | FULL TEXT  

Evaluation and Certification of Computerized Provider Order Entry Systems
Classen et al.
J. Am. Med. Inform. Assoc. 2007;14:48-55.
ABSTRACT | FULL TEXT  

Electronic Medical Records and Their Impact on Resident and Medical Student Education
Keenan et al.
Acad. Psychiatry 2006;30:522-527.
ABSTRACT | FULL TEXT  

Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial
Paul et al.
J Antimicrob Chemother 2006;58:1238-1245.
ABSTRACT | FULL TEXT  

The Stroke-Thrombolytic Predictive Instrument: A Predictive Instrument for Intravenous Thrombolysis in Acute Ischemic Stroke
Kent et al.
Stroke 2006;37:2957-2962.
ABSTRACT | FULL TEXT  

The Stroke-Thrombolytic Predictive Instrument Provides Valid Quantitative Estimates of Outcome Probabilities and Aids Clinical Decision-Making
Demaerschalk
Stroke 2006;37:2865-2866.
FULL TEXT  

Use of a Personal Digital Assistant for Managing Antibiotic Prescribing for Outpatient Respiratory Tract Infections in Rural Communities
Rubin et al.
J. Am. Med. Inform. Assoc. 2006;13:627-634.
ABSTRACT | FULL TEXT  

Effectiveness of Clinician-selected Electronic Information Resources for Answering Primary Care Physicians' Information Needs
McKibbon and Fridsma
J. Am. Med. Inform. Assoc. 2006;13:653-659.
ABSTRACT | FULL TEXT  

The Chronic Kidney Disease Epidemic: Stepping Back and Looking Forward
Kiberd
J. Am. Soc. Nephrol. 2006;17:2967-2973.
ABSTRACT | FULL TEXT