 |
|
Accessibility and Accuracy of Web Page References in 5 Major Medical Journals
To the Editor: Online access to information has changed the scope of medical research and data collection. Information formerly available only through many hours with Index Medicus and extensive correspondence is now easily obtained by most researchers. With this method of data collection becoming more widespread, Web site references are becoming more prevalent in leading medical journals.1 In 1984, prior to the proliferation of Internet references, de Lacey et al2 examined a random sample of citations from 6 medical journals and found an 8% major and a 24% overall error rate in citations referencing printed works. Twenty years later we investigated the accessibility and accuracy of Internet references for studies in 5 major medical journals 3 months after their publication.
Methods
In April 2004, we examined all original research papers that had appeared in the Annals of Internal Medicine, BMJ, JAMA, The Lancet, and the New England Journal of Medicine during the month of January 2004. We extracted all primary references to Web pages (references available from authors upon request). Uniform resource locators (URLs) were electronically cut and pasted from online full-text versions of the articles. These citations were categorized as accessible if the URL accessed a Web page, and as accurate if the information on that page was consistent with the content referenced in the evaluated article. Any URL that did not meet the accessibility criteria was then hand typed by 2 authors (R.C., N.W.) and rechecked on 3 separate computer networks. A URL was considered to have failed as an accessible citation only after failing by each method. Both reviewers separately evaluated each Internet reference for accuracy, with 100% concordance. For comparison, we also examined the citations in the de Lacey article for retrievability of printed copy through the local medical library or interlibrary loan.
Results
Ninety-one eligible articles were identified (TABLE). These articles included 68 URL references. Five (7.4%) of these were inaccurate and 3 (4.4%) were inaccessible within 3 months of publication. The total number of erroneous URL Web site references for each journal ranged from 0% to 22%, with a mean of 8.7% (95% confidence interval, 2.8%-20%).
|
|
|
|
Table. Eligible Articles, URL References, and Inaccessible or Inaccurate URL References
|
|
|
The de Lacey study cited 12 articles, ranging from 20 to 66 years old. Twenty years after publication of the de Lacey study we were unable to retrieve printed copy for 1 (8.3%) of these articles, comparable to the mean error rate of the 2004 URL citations at 3 months.
Comment
We found that within 3 months of publication, major medical journals have a number of URL references that are either inaccessible or inaccurate. This presents a problem for sustaining a durable research reference record. Regarding accessibility, how ephemeral the Web sites are is difficult to determine. Internet archivists have reported the average life span of a Web page to be 100 days.3 However, this includes governmental, educational, and consumer-based sites; medical and other scientific sites may have a different durability. It is also possible that typographical errors occurred in some of the reference lists. Regarding accuracy, one limitation of our study is that, because we were unable to measure the accuracy of each site at the moment of publication, we cannot determine whether informational discrepancy reflects errors related to the dynamic modification of the Web page over time or errors incorporated in the original article. This deserves further study.
There is no consensus regarding the need to archive a referenced Web page. If URL citations have printed counterparts it may be preferable to cite those instead. Saving either a printed or electronic copy of the Web page and making it available through the corresponding author would also be advisable. Implementation of archiving database solutions1 may become the responsibility of medical journals. Ultimately, the durability of references is essential to allow readers to retrieve and confirm information directly from cited primary sources.4
Access to Data: Dr Crichlow had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analyses.
Renée Crichlow, MD
reneec4{at}u.washington.edu
Stefanie Davies
University of Washington
Montana Family Medicine Residency Program
Billings
Nicole Winbush, MD
Saint Vincent Healthcare
Billings
1. Dellavalle RP, Hester EJ, Heilig LF, et al. Going, going, gone: lost Internet references. Science. 2003;302:787-788.
FREE FULL TEXT
2. de Lacey G, Record C, Wade J. How accurate are quotations and references in medical journals? BMJ. 1985;291:884-886.
ISI
| PUBMED
3. Mayfield K. Wayback Goes Way Back on Web. Wired News. Available at: http://www.wired.com/news/culture/0,1284,47894,00.html. Accessed May 14, 2004 and archived by primary author, 2004.
4. Siebers R, Holt S. Accuracy of references in five leading medical journals. Lancet. 2000;356:1445.
ISI
| PUBMED
Letters Section Editor: Robert M. Golub, MD, Senior Editor.
JAMA. 2004;292:2723-2724.
RELATED LETTERS
Permanence of Web Page References
Tamilarasu Kadhiravan
JAMA. 2005;293(12):1450.
EXTRACT
| FULL TEXT
Permanence of Web Page References—Reply
Renée Crichlow and Nicole Winbush
JAMA. 2005;293(12):1450.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
The Prevalence and Inaccessibility of Internet References in the Biomedical Literature at the Time of Publication
Aronsky et al.
J. Am. Med. Inform. Assoc. 2007;14:232-234.
ABSTRACT
| FULL TEXT
Scholarship Erosion
Badgett et al.
ANN INTERN MED 2006;145:77-77.
FULL TEXT
Permanence of Web Page References
Kadhiravan
JAMA 2005;293:1450-1450.
FULL TEXT
|
