This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (81)  • Contact me when this article is cited  Related Content  • Related letter  • Related article  • Similar articles in JAMA  Topic Collections  • Informatics/ Internet in Medicine  • Informatics, Other  • Review  • Alert me on articles by topic  Social Bookmarking   What's this?

A Framework for Investigation and Systematic Review

William R. Hersh, MD; David H. Hickam, MD, MPH

JAMA. 1998;280:1347-1352.

Objective.— Despite the proliferation of electronic information retrieval (IR) systems for physicians, their effectiveness has not been well assessed. The purpose of this review is to provide a conceptual framework and to apply the results of previous studies to this framework.

Data Sources.— All sources of medical informatics and information science literature, including MEDLINE, along with bibliographies of textbooks in these areas, were searched from 1966 to January 1998.

Study Selection.— All articles presenting either classifications of evaluation studies or their results, with an emphasis on those studying use by physicians.

Data Extraction.— A framework for evaluation was developed, consisting of frequency of use, purpose of use, user satisfaction, searching utility, search failure, and outcomes. All studies were then assessed based on the framework.

Data Synthesis.— Due to the heterogeneity and simplistic study designs, no meta-analysis of studies could be done. General conclusions were drawn from data where appropriate. A total of 47 articles were found to include an evaluation component and were used to develop the framework. Of these, 21 articles met the inclusion criteria for 1 or more of the categories in the framework. Most use of IR systems by physicians still occurs with bibliographic rather than full-text databases. Overall use of IR systems occurs just 0.3 to 9 times per physician per month, whereas physicians have 2 unanswered questions for every 3 patients.

Conclusions.— Studies comparing IR systems with different searching features have not shown that advanced searching methods are significantly more effective than simple text word methods. Most searches retrieve only one fourth to one half of the relevant articles on a given topic and, once retrieved, little is known about how these articles are interpreted or applied. These studies imply that further research and development are needed to improve system utility and performance.

From the Division of Medical Informatics & Outcomes Research (Drs Hersh and Hickam) and the Department of Medicine (Dr Hersh), Oregon Health Sciences University, and Health Services Research and Development, Department of Medicine, Portland VA Medical Center (Dr Hickam), Portland, Ore.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

RELATED LETTER

Electronic Information Retrieval by Physicians and Medical Librarians
Elizabeth Killoran and William Hersh
JAMA. 1999;281(14):1272-1273.
EXTRACT | FULL TEXT  

RELATED ARTICLE

Medical Information on the Internet
P. Robert Hubbs, Thomas C. Rindfleisch, Paul Godin, and Kenneth L. Melmon
JAMA. 1998;280(15):1363.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Effectiveness of Topic-specific Infobuttons: A Randomized Controlled Trial
Del Fiol et al.
J. Am. Med. Inform. Assoc. 2008;15:752-759.
ABSTRACT | FULL TEXT  

Healthcare personnel's use of e-information sources in Riyadh governmental hospitals
Khudair and Cooke
Journal of Librarianship and Information Science 2008;40:207-219.
ABSTRACT  

A Document Clustering and Ranking System for Exploring MEDLINE Citations
Lin et al.
J. Am. Med. Inform. Assoc. 2007;14:651-661.
ABSTRACT | FULL TEXT  

Assessment of internet use and effects among healthcare professionals: a cross sectional survey.
Podichetty et al.
Postgrad. Med. J. 2006;82:274-279.
ABSTRACT | FULL TEXT  

Using Citation Data to Improve Retrieval from MEDLINE
Bernstam et al.
J. Am. Med. Inform. Assoc. 2006;13:96-105.
ABSTRACT | FULL TEXT  

How computers can help to share understanding with patients
Sullivan and Wyatt
BMJ 2005;331:892-894.
FULL TEXT  

Do Online Information Retrieval Systems Help Experienced Clinicians Answer Clinical Questions?
Westbrook et al.
J. Am. Med. Inform. Assoc. 2005;12:315-321.
ABSTRACT | FULL TEXT  

Ubiquitous but Unfinished: Online Information Retrieval Systems
Hersh
Med Decis Making 2005;25:147-148.
 

The Impact of an Online Evidence System on Confidence in Decision Making in a Controlled Setting
Westbrook et al.
Med Decis Making 2005;25:178-185.
ABSTRACT  

Electronic Technology: A Spark to Revitalize Primary Care?
Bodenheimer and Grumbach
JAMA 2003;290:259-264.
ABSTRACT | FULL TEXT  

ISABEL: a web-based differential diagnostic aid for paediatrics: results from an initial performance evaluation
Ramnarayan et al.
Arch. Dis. Child. 2003;88:408-413.
ABSTRACT | FULL TEXT  

Medical Informatics: Improving Health Care Through Information
Hersh
JAMA 2002;288:1955-1958.
FULL TEXT  

Effects Of Current And Future Information Technologies On The Health Care Workforce
Masys
Health Aff (Millwood) 2002;21:33-41.
ABSTRACT | FULL TEXT  

Factors Associated with Success in Searching MEDLINE and Applying Evidence to Answer Clinical Questions
Hersh et al.
J. Am. Med. Inform. Assoc. 2002;9:283-293.
ABSTRACT | FULL TEXT  

Welcome, PIER, A New Physicians' Information and Education Resource
Badgett and Mulrow
ANN INTERN MED 2002;136:553-554.
FULL TEXT  

Using clinical evidence
Barton
BMJ 2001;322:503-504.
FULL TEXT  

The Informationist
Davidoff and Florance
ANN INTERN MED 2001;134:252-253.
FULL TEXT  

A Randomized Controlled Trial of Point-of-Care Evidence to Improve the Antibiotic Prescribing Practices for Otitis Media in Children
Christakis et al.
Pediatrics 2001;107:15e-15.
ABSTRACT | FULL TEXT  

Electronic Information Retrieval by Physicians and Medical Librarians
Killoran and Hersh
JAMA 1999;281:1272-1273.
FULL TEXT  

Medical Information on the Internet
Hubbs et al.
JAMA 1998;280:1363-1363.
FULL TEXT