This Article  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Topic Collections  • Aging/ Geriatrics  • Informatics/ Internet in Medicine  • Internet  • Patient-Physician Relationship/ Care  • Patient Education/ Health Literacy  • Alert me on articles by topic  Social Bookmarking   What's this?

To the Editor: The use of Internet-based applications to support health management tasks has grown enormously.¹ Studies investigating these applications have focused on prevalence of or reasons for use. We evaluated the ability of a sample of older adults to use the Medicare.gov Web site to make decisions concerning eligibility for services and prescription drug plans.

Methods
Two hundred eighteen people responded to newspaper advertisements and flyers in community and senior centers in the greater Miami area through telephone contact. Participants were required to be aged at least 50 years; be cognitively unimpaired (Mini-Mental State Examination² score >26); have had computer experience (assessed via questionnaire³); be English speaking; and not have depressive symptoms or severe visual or hearing impairments. Following screening, 201 were eligible for participation, 64 of whom expressed lack of interest or did not report for the study. Participants were enrolled consecutively; 112 completed the protocol and were compensated $50 (Table 1). To characterize the sample, data were collected on age, education, and race/ethnicity (based on self-report using fixed categories⁴). Data were collected at the University of Miami between February 2006 and May 2007. The study was approved by the university's institutional review board, and all participants provided written consent.

View this table: [in this window] [in a new window] [as a PowerPoint slide]   Table 1. Characteristics of the Study Sample

Participants were asked to use the Medicare.gov Web site to (1) determine eligibility for home health care services, (2) select a home health agency to meet specified needs, (3) make a decision about enrollment in Medicare Part D based on specified criteria, and (4) select a drug plan and determine associated costs based on a specified medication regime. They provided written responses. Participants also evaluated Web site usability and the value of Internet health information via questionnaire. Responses were rated by 2 investigators using a scoring sheet and through analysis of video records.⁶ Interrater reliability on 2 problems in a random sample of 20 participants was good (r = 0.96 and r = 0.97).

During data collection, changes were made to the appearance of the home page and sizing of selection buttons on the Web site; content was not altered. To assess comparability, we compared performance scores, usability ratings, and characteristics of participants who used the initial (n = 82) and modified (n = 30) versions using ² tests, t tests, and Wilcoxon tests. No differences were found for ratings of usability or performance scores. A greater percentage of Hispanic (46.2%) and non-Hispanic white (44.9%) participants used the initial version than non-Hispanic black participants (9.0%), but more non-Hispanic black participants (42.9%) used the modified version than Hispanic (28.6%) or non-Hispanic white participants (28.6%) (P < .001). Analyses were performed using SPSS 15.0 (SPSS Inc, Chicago, Illinois), with 2-sided significance set at P < .05.

Results
Most participants were unable to specify all eligibility criteria for home health services (n = 77 [68.8%; 95% confidence interval {CI}, 59.8%-76.8%]), choose the correct home health agency (n = 90 [80.4%; 95% CI, 72.3%-86.9%]), or execute computation procedures needed for making a plan enrollment decision (n = 94 [83.9%; 95% CI, 76.3%-89.8%]). Only about half (n = 64 [57.1%; 95% CI, 47.9%-66.0%]) were able to make an enrollment decision. Most participants (n = 81 [72.3%; 95% CI, 63.5%-80.0%]) were unable to select a drug plan or had problems navigating to the necessary Web pages, locating needed information, or executing actions required for plan selection. Participants also reported problems with usability (Table 2), but almost all indicated that the Internet was a valuable source of health information.

View this table: [in this window] [in a new window] [as a PowerPoint slide]   Table 2. Task Performance and Usability Ratings (n = 112)

Comment
A sample of relatively well-educated adults with computer experience had difficulty using the Medicare.gov Web site to determine eligibility for services and enroll in a drug plan. Participants also reported problems navigating within the Web site and locating information. Although the sample was relatively small and restricted to people aged 50 years or more, it was diverse and larger than samples included in most usability studies. It is likely that persons with less computer experience would have even greater difficulty. To ensure that electronic health tools reach their full potential, broad and inclusive input from consumers should serve as the basis for design.

Author Contributions: Mr Nair had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Czaja, Sharit, Nair.

Acquisition of data: Czaja.

Analysis and interpretation of data: Czaja, Sharit, Nair.

Drafting of the manuscript: Czaja, Nair.

Critical revision of the manuscript for important intellectual content: Czaja, Sharit.

Statistical analysis: Czaja, Nair.

Obtained funding: Czaja, Sharit.

Administrative, technical, or material support: Nair.

Study supervision: Czaja, Sharit.

Financial Disclosures: None reported.

Funding Support: This research was supported by grant 2 P01 AG017211 from the National Institutes of Health/National Institute on Aging.

Role of the Sponsor: The National Institutes of Health played no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

Additional Contributions: Tamar El-Attar, MS, University of Miami Department of Industrial Engineering, and Mario Hernandez, MS, Center on Aging, University of Miami, both provided assistance in data collection. Chin Chin Lee, MS, MSPH, Center on Aging, University of Miami, provided administrative assistance and study supervision. Shih Hua Fu, MS, Center on Aging, University of Miami, provided technical assistance. No one received compensation for their roles.

Sara J. Czaja, PhD
sczaja{at}med.miami.edu
Department of Psychiatry and Behavioral Sciences
University of Miami Miller School of Medicine
Miami, Florida

Joseph Sharit, PhD
Department of Industrial Engineering
University of Miami

Sankaran N. Nair, MS
Center on Aging
University of Miami Miller School of Medicine

1. Expanding the reach and impact of consumer e-Health tools. US Department of Health and Human Services. http://www.health.gov/communication/ehealth/ehealthTools/executivesummary.htm. Accessed July 24, 2008. 2. Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-198. FULL TEXT | ISI | PUBMED 3. Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Computer and technology experience questionnaire [CREATE Technical Rep No. CREATE-2006-03]. 2006.4. Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Demographic and background questionnaire [CREATE Technical Rep No. CREATE-2006-02]. 2006.5. Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38(1):33-42. FULL TEXT | ISI | PUBMED 6. HyperCam [computer program]. Murrysville, PA: Hyperionics Technology; 2006.

Letters Section Editor: Robert M. Golub, MD, Senior Editor.

JAMA. 2008;300(7):790-792.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?