This Article  • Full text  • PDF  • Online Tables  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (226)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Informatics/ Internet in Medicine  • Internet  • Patient-Physician Relationship/ Care  • Patient Education/ Health Literacy  • Review  • Alert me on articles by topic  Social Bookmarking   What's this?

A Systematic Review

Gunther Eysenbach, MD; John Powell, MSc, MRCPsych, MFPHM; Oliver Kuss, PhD; Eun-Ryoung Sa, MS

JAMA. 2002;287:2691-2700.

Context  The quality of consumer health information on the World Wide Web is an important issue for medicine, but to date no systematic and comprehensive synthesis of the methods and evidence has been performed.

Objectives  To establish a methodological framework on how quality on the Web is evaluated in practice, to determine the heterogeneity of the results and conclusions, and to compare the methodological rigor of these studies, to determine to what extent the conclusions depend on the methodology used, and to suggest future directions for research.

Data Sources  We searched MEDLINE and PREMEDLINE (1966 through September 2001), Science Citation Index (1997 through September 2001), Social Sciences Citation Index (1997 through September 2001), Arts and Humanities Citation Index (1997 through September 2001), LISA (1969 through July 2001), CINAHL (1982 through July 2001), PsychINFO (1988 through September 2001), EMBASE (1988 through June 2001), and SIGLE (1980 through June 2001). We also conducted hand searches, general Internet searches, and a personal bibliographic database search.

Study Selection  We included published and unpublished empirical studies in any language in which investigators searched the Web systematically for specific health information, evaluated the quality of Web sites or pages, and reported quantitative results. We screened 7830 citations and retrieved 170 potentially eligible full articles. A total of 79 distinct studies met the inclusion criteria, evaluating 5941 health Web sites and 1329 Web pages, and reporting 408 evaluation results for 86 different quality criteria.

Data Extraction  Two reviewers independently extracted study characteristics, medical domains, search strategies used, methods and criteria of quality assessment, results (percentage of sites or pages rated as inadequate pertaining to a quality criterion), and quality and rigor of study methods and reporting.

Data Synthesis  Most frequently used quality criteria used include accuracy, completeness, readability, design, disclosures, and references provided. Fifty-five studies (70%) concluded that quality is a problem on the Web, 17 (22%) remained neutral, and 7 studies (9%) came to a positive conclusion. Positive studies scored significantly lower in search (P = .02) and evaluation (P = .04) methods.

Conclusions  Due to differences in study methods and rigor, quality criteria, study population, and topic chosen, study results and conclusions on health-related Web sites vary widely. Operational definitions of quality criteria are needed.

Author Affiliations: Unit for Cybermedicine and E-Health, Department of Clinical Social Medicine, University of Heidelberg, Heidelberg, Germany (Dr Eysenbach); Health Services Research Unit, London School of Hygiene and Tropical Medicine, London, England (Dr Powell); Department of Medical Epidemiology, Biometry and Informatics, University of Halle-Wittenberg, Halle/Saale, Germany (Dr Kuss); and Global Health Network Group, Department of Epidemiology, University of Pittsburgh, Pittsburgh, Pa (Ms Sa). Dr Eysenbach is now with the Centre for Global eHealth Innovation, Toronto General Hospital, Toronto, Ontario.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

RELATED ARTICLES

Health Information on the Internet: Quality Issues and International Initiatives
Ahmad Risk and Carolyn Petersen
JAMA. 2002;287(20):2713-2715.
EXTRACT | FULL TEXT  

May 22/29, 2002
JAMA. 2002;287(20):2723-2724.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Genital herpes: an Internet-based risk survey
Mindel et al.
Int J STD AIDS 2009;20:785-789.
ABSTRACT | FULL TEXT  

Surfing for Juvenile Idiopathic Arthritis: Perspectives on Quality and Content of Information on the Internet
STINSON et al.
The Journal of Rheumatology 2009;36:1755-1762.
ABSTRACT | FULL TEXT  

Patients' views of a multimedia resource featuring experiences of rheumatoid arthritis: pilot evaluation of www.healthtalkonline.org
Newman et al.
Health Informatics Journal 2009;15:147-159.
ABSTRACT  

Where's the Revolution? Digital Technology and Health Care in the Internet Age
Miller and West
Journal of Health Politics, Policy and Law 2009;34:261-284.
ABSTRACT  

Computer and Internet Use in a Community Health Clinic Population
Peterson et al.
Med Decis Making 2009;29:202-206.
ABSTRACT  

Patient-Oriented Methotrexate Information Sites on the Internet: A Review of Completeness, Accuracy, Format, Reliability, Credibility, and Readability
THOMPSON and GRAYDON
The Journal of Rheumatology 2009;36:41-49.
ABSTRACT | FULL TEXT  

Health information for patients: time to look beyond patient information leaflets
Colledge et al.
JRSM 2008;101:447-453.
FULL TEXT  

The quality of nutritional information available on popular websites: a content analysis
Ostry et al.
Health Educ Res 2008;23:648-655.
ABSTRACT | FULL TEXT  

Characterisation and evaluation of UK websites on attention deficit hyperactivity disorder
Akram et al.
Arch. Dis. Child. 2008;93:695-700.
ABSTRACT | FULL TEXT  

Developing Informatics Tools and Strategies for Consumer-centered Health Communication
Keselman et al.
J. Am. Med. Inform. Assoc. 2008;15:473-483.
ABSTRACT | FULL TEXT  

Reasons, assessments and actions taken: sex and age differences in uses of Internet health information
Ybarra and Suman
Health Educ Res 2008;23:512-521.
ABSTRACT | FULL TEXT  

Quality of information accompanying on-line marketing of home diagnostic tests
Datta et al.
JRSM 2008;101:34-38.
ABSTRACT | FULL TEXT  

Quality of anaesthesia-related information accessed via Internet searches
Caron et al.
Br J Anaesth 2007;99:195-201.
ABSTRACT | FULL TEXT  

Using online databases to find peer-reviewed journal articles on injury prevention and safety promotion research: a study of textword queries by SafetyLit users
Lawrence
Inj. Prev. 2007;13:232-236.
ABSTRACT | FULL TEXT  

The role of communication in paediatric drug safety
Stebbing et al.
Arch. Dis. Child. 2007;92:440-445.
ABSTRACT | FULL TEXT  

Health information: does quality count for the consumer?: How consumers evaluate the quality of health information materials across a variety of media
Marshall and Williams
Journal of Librarianship and Information Science 2006;38:141-156.
ABSTRACT  

Complementary and Alternative Medicine During Cancer Treatment: Beyond Innocence
Tascilar et al.
The Oncologist 2006;11:732-741.
ABSTRACT | FULL TEXT  

Informed Patients and the Internet: A Mediated Context for Consultations with Health Professionals
Kivits
J Health Psychol 2006;11:269-282.
ABSTRACT  

Suitability of written supplemental materials available on the Internet for nonprescription medications
Wallace et al.
Am J Health Syst Pharm 2006;63:71-78.
ABSTRACT | FULL TEXT  

More information, more choice: an Italian database for oncology patients
Bogliolo et al.
Ann Oncol 2005;16:1962-1967.
ABSTRACT | FULL TEXT  

Using the Internet to Provide Information Prescriptions
Ritterband et al.
Pediatrics 2005;116:e643-e647.
ABSTRACT | FULL TEXT  

Harnessing the potential of the Internet to promote chronic illness self-management: diabetes as an example of how well we are doing
Bull et al.
Chronic Illness 2005;1:143-155.
ABSTRACT  

Commentary: Patient-Centered Health Information Technology
Cain and Clancy
American Journal of Medical Quality 2005;20:164-166.
 

Promoting Health Literacy
McCray
J. Am. Med. Inform. Assoc. 2005;12:152-163.
ABSTRACT | FULL TEXT  

Patient Internet Use for Health Information at Three Urban Primary Care Clinics
Dickerson et al.
J. Am. Med. Inform. Assoc. 2004;11:499-504.
ABSTRACT | FULL TEXT  

Design of Genetics Home Reference: A New NLM Consumer Health Resource
Mitchell et al.
J. Am. Med. Inform. Assoc. 2004;11:439-447.
ABSTRACT | FULL TEXT  

Implications for Designing Online Oral Health Resources: A Review of Fifty-Six Websites
Kim et al.
J Dent Educ 2004;68:633-643.
ABSTRACT | FULL TEXT  

Assessing websites on complementary and alternative medicine for cancer
Schmidt and Ernst
Ann Oncol 2004;15:733-742.
ABSTRACT | FULL TEXT  

Websites on screening for breast cancer: "Infodemiology" studies have surely had their day
Eysenbach
BMJ 2004;328:769-769.
FULL TEXT  

How the internet affects patients' experience of cancer: a qualitative study
Ziebland et al.
BMJ 2004;328:564.
ABSTRACT | FULL TEXT  

Ophthalmology on the Internet: What Do Our Patients Find?
Kahana and Gottlieb
Arch Ophthalmol 2004;122:380-382.
ABSTRACT | FULL TEXT  

The Impact of the Internet on Cancer Outcomes
Eysenbach
CA Cancer J Clin 2003;53:356-371.
ABSTRACT | FULL TEXT  

Emerging challenges in using health information from the internet
Theodosiou and Green
Adv. Psychiatr. Treat. 2003;9:387-396.
ABSTRACT | FULL TEXT  

Electronic Technology: A Spark to Revitalize Primary Care?
Bodenheimer and Grumbach
JAMA 2003;290:259-264.
ABSTRACT | FULL TEXT  

Psychosocial Support for Patients in Pediatric Oncology: The Influences of Parents, Schools, Peers, and Technology
Suzuki and Kato
Journal of Pediatric Oncology Nursing 2003;20:159-174.
ABSTRACT  

The Internet for Medical Information About Cancer: Help or Hindrance?
Matthews et al.
Psychosomatics 2003;44:100-103.
ABSTRACT | FULL TEXT  

The doctor, the patient and the world-wide web: how the internet is changing healthcare
Powell et al.
JRSM 2003;96:74-76.
FULL TEXT  

Health Information on the Internet: Quality Issues and International Initiatives
Risk and Petersen
JAMA 2002;287:2713-2715.
FULL TEXT