 |
|
Persistence of Contradicted Claims in the Literature
Athina Tatsioni, MD;
Nikolaos G. Bonitsis, MD;
John P. A. Ioannidis, MD
JAMA. 2007;298(21):2517-2526.
Context Some research findings based on observational epidemiology are contradicted by randomized trials, but may nevertheless still be supported in some scientific circles.
Objectives To evaluate the change over time in the content of citations for 2 highly cited epidemiological studies that proposed major cardiovascular benefits associated with vitamin E in 1993; and to understand how these benefits continued being defended in the literature, despite strong contradicting evidence from large randomized clinical trials (RCTs). To examine the generalizability of these findings, we also examined the extent of persistence of supporting citations for the highly cited and contradicted protective effects of beta-carotene on cancer and of estrogen on Alzheimer disease.
Data Sources For vitamin E, we sampled articles published in 1997, 2001, and 2005 (before, early, and late after publication of refuting evidence) that referenced the highly cited epidemiological studies and separately sampled articles published in 2005 and referencing the major contradicting RCT (HOPE trial). We also sampled articles published in 2006 that referenced highly cited articles proposing benefits associated with beta-carotene for cancer (published in 1981 and contradicted long ago by RCTs in 1994-1996) and estrogen for Alzheimer disease (published in 1996 and contradicted recently by RCTs in 2004).
Data Extraction The stance of the citing articles was rated as favorable, equivocal, and unfavorable to the intervention. We also recorded the range of counterarguments raised to defend effectiveness against contradicting evidence.
Results For the 2 vitamin E epidemiological studies, even in 2005, 50% of citing articles remained favorable. A favorable stance was independently less likely in more recent articles, specifically in articles that also cited the HOPE trial (odds ratio for 2001, 0.05 [95% confidence interval, 0.01-0.19; P < .001] and the odds ratio for 2005, 0.06 [95% confidence interval, 0.02-0.24; P < .001], as compared with 1997), and in general/internal medicine vs specialty journals. Among articles citing the HOPE trial in 2005, 41.4% were unfavorable. In 2006, 62.5% of articles referencing the highly cited article that had proposed beta-carotene and 61.7% of those referencing the highly cited article on estrogen effectiveness were still favorable; 100% and 96%, respectively, of the citations appeared in specialty journals; and citations were significantly less favorable (P = .001 and P = .009, respectively) when the major contradicting trials were also mentioned. Counterarguments defending vitamin E or estrogen included diverse selection and information biases and genuine differences across studies in participants, interventions, cointerventions, and outcomes. Favorable citations to beta-carotene, long after evidence contradicted its effectiveness, did not consider the contradicting evidence.
Conclusion Claims from highly cited observational studies persist and continue to be supported in the medical literature despite strong contradictory evidence from randomized trials.
Author Affiliations: Department of Hygiene and Epidemiology, (Drs Tatsioni, Bonitsis, and Ioannidis) and the Department of Dermatology (Dr Bonitsis), University of Ioannina School of Medicine; and the Biomedical Research Institute, Foundation for Research and Technology-Hellas (Dr Ioannidis), Ioannina, Greece; Institute for Clinical Research and Health Policy Studies, Department of Medicine, Tufts University School of Medicine, Boston, Massachusetts (Drs Tatsioni and Ioannidis).
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
The Author Responds to "Evaluating p Values and Bayes Factors"
Ioannidis
Am J Epidemiol 2008;168:389-390.
FULL TEXT
Everyone's a Little Bit Biased (Even Physicians)
Cain and Detsky
JAMA 2008;299:2893-2895.
FULL TEXT
Giving antioxidants to infants with Down's syndrome
Reynolds
BMJ 2008;336:568-569.
FULL TEXT
What's new in the other general journals
BMJ 2007;335:1234-1235.
FULL TEXT
|
