 |
|
Value of Flow Diagrams in Reports of Randomized Controlled Trials
Matthias Egger, MD;
Peter Jüni, MD;
Christopher Bartlett, PhD;
for the CONSORT Group
JAMA. 2001;285:1996-1999.
ABSTRACT
| |
Context Diagrams of the flow of participants through a clinical trial are recommended in the Consolidated Standards for Reporting of Trials (CONSORT) statement, but it is unclear whether such flow diagrams improve the quality of trial reports.
Objective To examine the information contributed by flow diagrams and the completeness of reporting overall in reports of randomized controlled trials (RCTs) published in 5 general and internal medicine journals.
Design and Setting Analysis of 270 reports of RCTs published in 1998 in the Annals of Internal Medicine (AIM; n = 19), BMJ (n = 42), JAMA (n = 45), The Lancet (n = 81), and The New England Journal of Medicine (NEJM; n = 83).
Main Outcome Measures Proportion of reports that included a flow diagram, information provided in flow diagrams, and completeness of reporting about flow of participants overall in flow diagrams or text.
Results A total of 139 reports (51.5%) of RCTs included a flow diagram, but this varied widely among journals (AIM, 21.0%; BMJ, 38.1%; JAMA, 80.0%; The Lancet, 93.8%; and NEJM, 8.4%). Diagrams generally provided useful information, but only 73 (52.5%) included the number of participants who received allocated interventions and only 32 (23.0%) included the number of participants included in the analysis. In logistic regression analysis, overall completeness of reporting about flow of study participants was associated with publication of a flow diagram.
Conclusions Flow diagrams are associated with improved quality of reporting of randomized controlled trials. However, the structure of current flow diagrams is less than ideal. We propose a revised flow diagram that includes all important counts through the stages of parallel group trials.
INTRODUCTION
The Consolidated Standards for Reporting of Trials (CONSORT) criteria were published in 19961 to assist authors in reporting randomized controlled trials (RCTs). The CONSORT statement consists of a checklist for items in the methods, results, and discussion sections of a trial report and a diagram to show the flow of participants through the various stages of the trial (Figure 1, A). The CONSORT statement has been endorsed by an increasing number of journals,2-5 but it is unclear whether the use of CONSORT has improved the quality of reports of RCTs. We examined to what extent 5 leading medical journals adopted flow diagrams, analyzed the information contributed by these diagrams, and assessed completeness of reporting overall. The results guided our attempt to improve the design of the flow diagram (Figure 1, B).
|
|
|
|
Figure. Original and Proposed Revised Versions of CONSORT Flow Chart
|
|
|
METHODS
One of us (C.B.) searched each issue published in 1998 of the Annals of Internal Medicine (AIM), BMJ, JAMA, The Lancet, and The New England Journal of Medicine (NEJM) for published reports of RCTs. A study was defined as an RCT if the assignment of participants to interventions was described as randomized by words such as randomly, random, and randomization. Two of us (M.E., P.J.) independently extracted data on the characteristics of reports and examined whether a flow diagram was included. Articles were examined in random sequence with 1 assessor examining trials in the opposite order. We searched each journal's Web site for diagrams that were published electronically but did not appear in print.
We examined whether 6 counts were provided in diagrams: (1) number of patients assessed for eligibility, (2) number found to be eligible, (3) number randomized (per group), (4) number who received allocated intervention (per group), (5) number who were lost to follow-up (per group), and (6) number included in the main analysis of primary outcomes (per group). In a separate step, we assessed whether these counts were provided either in a flow diagram or anywhere else in the article.
Each article was read independently by 2 of us in an incomplete randomized Latin square design, and interrater reliability6 for assessment of information in flow diagrams and overall was determined. Disagreements were resolved by consensus. We compared overall completeness of reporting between journals using  2 tests and used logistic regression to assess the association of flow diagrams with completeness of reporting.
RESULTS
The hand search of these journals identified 290 articles. We excluded 1 quasi-randomized trial, 10 cluster trials, and 9 reports for which the focus was not on randomized comparisons. Our study sample thus consisted of 270 reports of RCTs. Most trials were of parallel group design (256 [94.8%]) and evaluated pharmacological interventions (173 [64.1%]). A total of 139 reports (51.5%) included a flow diagram but this varied widely among journals (Table 1). Five (29.4%) of 17 short reports included a flow diagram. No additional diagrams were found on the journals' Web sites. Interrater reliability6 for the assessment of information provided in flow diagrams was good ( values ranging from 0.60 to 0.81), but showed more disagreement for assessment of overall completeness of reporting (, 0.21-0.71).
|
|
|
|
Table 1. Information on Participant Flow Provided in 270 Reports of Randomized Controlled Trials Published in 5 Journals in 1998*
|
|
|
Characteristics of flow diagrams were similar across journals. Sixty-five diagrams (46.8%) included the number of patients assessed for eligibility, and 94 (67.6%) reported the number found to be eligible. The number assigned to each study group was reported in 129 flow diagrams (92.8%), and 73 (52.5%) indicated whether interventions had been received as allocated. Most flow diagrams (114 [82.0%]) indicated how many patients in each group were lost to follow-up, but only 32 (23.0%) reported the number included in the analysis.
Overall reporting was more complete in journals that publish flow diagrams more frequently (Table 1). However, there is clearly room for improvement in all journals, in particular with regard to the number receiving the allocated intervention and the number lost to follow-up. Independent of journal, reporting tended to be less complete if the trial was published as a short report. In logistic regression analysis, completeness of reporting was associated with the publication of a flow diagram (Table 2). In multivariable analyses, there was little evidence for confounding by journal, and differences among journals were largely explained by the publication or nonpublication of flow diagrams.
|
|
|
|
Table 2. Association of Presence of a Flow Diagram With Completeness of Reporting*
|
|
|
COMMENT
A diagram showing the flow of participants from enrollment to analysis is an important element of the CONSORT standards for the reporting of clinical trials. Flow diagrams provide an aid to trialists when writing trial results and assist readers in the critical appraisal of the internal and external validity of a trial. We found that the information on the progress of participants through the trial was more complete in journals that frequently publish flow diagrams and that the publication of these diagrams is associated with better reporting in individual articles. Our results are consistent with those of Moher and colleagues,7 who compared reports published in the same journals before and after the CONSORT statement became available. The proportion of articles that included a flow diagram varied widely across journals and was lower in short reports. Flow diagrams take up precious journal space, and editors may sometimes feel that this space is better used otherwise. Our findings provide strong support for the idea that RCTs always should be published as full articles8 including a flow diagram.
The shortcomings of the flow diagram template recommended in 19961 may be another reason some editors are reluctant to publish flow diagrams. Meinert9 pointed out that the terms used in the 1996 CONSORT statement lacked clarity and that the information presented in the flow diagram was incomplete. Our results indicate that there were problems with both the original design of the flow diagram and its implementation by authors. For example, most flow diagrams provided the number of individuals randomized, although this count was not explicitly requested.9 Conversely, only about half of flow diagrams included the number of participants who actually received treatments as allocated, an item included in the original template.1 The number of participants included in the main analysis was not an item in the recommended flow diagram, and this number was included in only a few diagrams (23.0%). This finding is of concern because the latter count is essential for appraising whether a trial has been analyzed by intention to treat. A recent study found that intention-to-treat approaches are often inadequately described and inadequately applied.10 The number of persons assessed for eligibility was also frequently missing. Although this number is not relevant for assessing the internal validity of a trial, it is useful to estimate whether trial participants were likely to be representative of all patients seen.11
The CONSORT criteria are an evolving tool designed to help improve the quality of reporting of RCTs.12 Based on the findings of our present study and earlier criticism of CONSORT,9 we submit that the flow diagram is useful but in need of revision. We revised the template in collaboration with the CONSORT Group (Figure 1).13 Three tiers of boxes cover the enrollment phase and 1 box each is set apart for allocation of interventions, follow-up, and analysis. At enrollment a distinction is made between the exclusion of persons not meeting the criteria specified in the protocol and persons excluded for other reasons. The revised flow diagram explicitly includes the number of participants allocated to each group, the number initially receiving the intervention as assigned, and, if applicable, the reasons why some participants did not receive allocated interventions. Information on follow-up includes the number of participants lost to follow-up and the number of patients who stopped interventions along with the reasons why this occurred. The bottom tier of boxes includes the number of patients included in the main analysis and, if applicable, the reasons why some patients were excluded. We trust that this template could be improved further and would appreciate comments from readers through the CONSORT Web site.14
AUTHOR INFORMATION
Author Contributions: Study concept and design: Egger, Jüni.
Acquisition of data: Egger, Jüni, Bartlett.
Analysis and interpretation of data: Egger, Jüni.
Drafting of the manuscript: Egger.
Critical revision of the manuscript for important intellectual content: Egger, Jüni, Bartlett.
Statistical expertise: Egger, Jüni.
Administrative, technical, or material support: Egger, Bartlett.
Study supervision: Egger.
Funding/Support: The Ottawa meeting was made possible by funding from Abbott Laboratories, American College of Physicians, GlaxoWellcome, National Library of Medicine, The Lancet, Merck, and the Medical Research Council of Canada. Drs Jüni and Egger are supported by the Swiss National Science Foundation and the United Kingdom's Medical Research Council Health Services Research Collaboration.
Acknowledgment: We thank all members of the following groups: Standards for Reporting Trials (SORT), Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature, and the CONSORT Group for their support and encouragement in helping to bring the CONSORT statement to its present form. We are grateful to Frank Davidoff, MD, Peter Gøtzsche, MD, Barbara Hawkins, PhD, Nicola Low, MD, Curt Meinert, PhD, David Moher, MSc, and Tim Peters, PhD, for helpful comments on an earlier version of the manuscript. The work reported herein was prompted by discussions during the second meeting of the CONSORT Group in Ottawa, Ontario, in May 1999. The members of the CONSORT Group who participated in the Ottawa meeting were Doug Altman, PhD, Jesse Berlin, PhD, Frank Davidoff, MD, Matthias Egger, MD, Diana Elbourne, PhD, Peter Gøtzsche, MD, Sylvan Green, PhD, Barbara Hawkins, PhD, Richard Horton, MD, Terry Klassen, MD, Tom Lang, MA, Curt Meinert, PhD, David Moher, MSc, Drummond Rennie, MD, Ken Schulz, PhD, and Bruce Squires, MD.
Corresponding Author and Reprints: Matthias Egger, MD, Medical Research Council Health Services Research Collaboration, Department of Social Medicine, Canynge Hall, Whiteladies Road, Bristol BS8 2PR, England (e-mail: m.egger{at}bristol.ac.uk).
Author Affiliations: Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol, England (Drs Egger, Jüni, and Bartlett); and Department of Social and Preventive Medicine, University of Berne, Berne, Switzerland (Drs Egger and Jüni).
REFERENCES
| |
1. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA. 1996;276:637-639.
FREE FULL TEXT
2. Altman DG. Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996;313:570-571.
FREE FULL TEXT
3. A standard method for reporting randomized medical scientific research: the "consolidation of the standards of reporting trials" (CONSORT) [in Dutch]. Ned Tijdschr Geneeskd. 1998;142:1089-1091.
PUBMED
4. Huston P, Hoey J. CMAJ endorses the CONSORT statement: Consolidation of Standards for Reporting Trials. CMAJ. 1996;155:1277-1282.
PUBMED
5. Ausejo M, Saenz A, Moher D. CONSORT: an attempt to improve the quality of publication of clinical trials [in Spanish]. Aten Primaria. 1998;21:351-352.
PUBMED
6. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-174.
FULL TEXT
|
ISI
| PUBMED
7. Moher D, Jones A, LePage L for the CONSORT Group. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285:1992-1995.
FREE FULL TEXT
8. Deeks JJ, Altman DG. Inadequate reporting of controlled trials as short reports [letter]. Lancet. 1998;352:1908.
FULL TEXT
|
ISI
| PUBMED
9. Meinert CL. Beyond CONSORT: need for improved reporting standards for clinical trials. JAMA. 1998;279:1487-1489.
FREE FULL TEXT
10. Hollis S, Campbell F. What is meant by intention to treat analysis? survey of published randomised controlled trials. BMJ. 1999;319:670-674.
FREE FULL TEXT
11. Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. In: Egger M, Smith DG, Altman DG, eds. Systematic Reviews in Health Care: Meta-analysis in Context. London, England: BMJ Publishing Group; 2001.
12. Moher D. CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. JAMA. 1998;279:1489-1491.
FREE FULL TEXT
13. Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285:1987-1991.
FREE FULL TEXT
14. CONSORT Web site. Available at: http://consort-statement.org. Accessed September 6, 2000.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati
What's this?
RELATED ARTICLES
The CONSORT Statement: Revised Recommendations for Improving the Quality of Reports of Parallel-Group Randomized Trials
David Moher, Kenneth F. Schulz, Douglas Altman, and for the CONSORT Group
JAMA. 2001;285(15):1987-1991.
ABSTRACT
| FULL TEXT
Use of the CONSORT Statement and Quality of Reports of Randomized Trials: A Comparative Before-and-After Evaluation
David Moher, Alison Jones, Leah Lepage, and for the CONSORT Group
JAMA. 2001;285(15):1992-1995.
ABSTRACT
| FULL TEXT
Physician Interpretations and Textbook Definitions of Blinding Terminology in Randomized Controlled Trials
P. J. Devereaux, Braden J. Manns, William A. Ghali, Hude Quan, Christina Lacchetti, Victor M. Montori, Mohit Bhandari, and Gordon H. Guyatt
JAMA. 2001;285(15):2000-2003.
ABSTRACT
| FULL TEXT
CONSORT Revised—Improving the Reporting of Randomized Trials
Drummond Rennie
JAMA. 2001;285(15):2006-2007.
EXTRACT
| FULL TEXT
April 18, 2001
JAMA. 2001;285(15):2023-2024.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Publication guidelines for quality improvement studies in health care: evolution of the SQUIRE project
Davidoff et al.
BMJ 2009;338:a3152-a3152.
ABSTRACT
| FULL TEXT
Publication Guidelines for Improvement Studies in Health Care: Evolution of the SQUIRE Project
Davidoff et al.
ANN INTERN MED 2008;149:670-676.
ABSTRACT
| FULL TEXT
Publication guidelines for quality improvement in health care: evolution of the SQUIRE project
Davidoff et al.
Qual Saf Health Care 2008;17:i3-i9.
ABSTRACT
| FULL TEXT
The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration
Ogrinc et al.
Qual Saf Health Care 2008;17:i13-i32.
ABSTRACT
| FULL TEXT
Quality of Reporting of Randomized Controlled Trials in General Endocrinology Literature
Rios et al.
J. Clin. Endocrinol. Metab. 2008;93:3810-3816.
ABSTRACT
| FULL TEXT
Randomized Clinical Trial of Small-Incision and Laparoscopic Cholecystectomy in Patients With Symptomatic Cholecystolithiasis: Primary and Clinical Outcomes
Keus et al.
Arch Surg 2008;143:371-377.
ABSTRACT
| FULL TEXT
Evaluating the Quality of Randomized Controlled Trials that Examine the Efficacy of Natural Health Products: A Systematic Review of Critical Appraisal Instruments
Whelan et al.
Evid Based Complement Alternat Med 2008;0:nem186v1-nem186.
ABSTRACT
| FULL TEXT
Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies
Elm et al.
BMJ 2007;335:806-808.
FULL TEXT
The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies
von Elm et al.
ANN INTERN MED 2007;147:573-577.
ABSTRACT
| FULL TEXT
Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration
Vandenbroucke et al.
ANN INTERN MED 2007;147:W-163-W-194.
ABSTRACT
| FULL TEXT
Reporting Implementation in Randomized Trials: Proposed Additions to the Consolidated Standards of Reporting Trials Statement
Mayo-Wilson
Am. J. Public Health 2007;97:630-633.
ABSTRACT
| FULL TEXT
"Surgical research or comic opera" redux.
Sade
Ann. Thorac. Surg. 2006;82:1173-1174.
FULL TEXT
Documenting research in scientific articles: guidelines for authors: reporting research designs and activities.
Lang
Chest 2006;130:1263-1268.
FULL TEXT
Improving the quality of reporting randomized controlled trials in cardiothoracic surgery: The way forward.
Tiruvoipati et al.
J. Thorac. Cardiovasc. Surg. 2006;132:233-240.
ABSTRACT
| FULL TEXT
Systematic Evaluation of the Quality of Randomized Controlled Trials in Diabetes
Montori et al.
Diabetes Care 2006;29:1833-1838.
ABSTRACT
| FULL TEXT
Bias in Clinical Intervention Research
Gluud
Am J Epidemiol 2006;163:493-501.
ABSTRACT
| FULL TEXT
Reporting of Noninferiority and Equivalence Randomized Trials: An Extension of the CONSORT Statement
Piaggio et al.
JAMA 2006;295:1152-1160.
ABSTRACT
| FULL TEXT
Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors
Altman and for the CONSORT Group
BMJ 2005;330:1056-1057.
FULL TEXT
Quality of Reporting of Diagnostic Accuracy Studies
Smidt et al.
Radiology 2005;235:347-353.
ABSTRACT
| FULL TEXT
The quality of reporting of diagnostic accuracy studies published in ophthalmic journals
Siddiqui et al.
Br. J. Ophthalmol. 2005;89:261-265.
ABSTRACT
| FULL TEXT
Quality of Reporting of Observational Longitudinal Research
Tooth et al.
Am J Epidemiol 2005;161:280-288.
ABSTRACT
| FULL TEXT
Investigating patient exclusion bias in meta-analysis
Tierney and Stewart
Int J Epidemiol 2005;34:79-87.
ABSTRACT
| FULL TEXT
The Consolidated Standards of Reporting Trials (CONSORT): Better Presentation of Surgical Trials in the Journal of Hand Surgery
SAUERLAND and DAVIS
J Hand Surg Eur Vol 2004;29:621-624.
ABSTRACT
| FULL TEXT
Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond
Moher et al.
CMAJ 2004;171:349-350.
FULL TEXT
Randomized Clinical Trials: How Long Should They Last?
Wellens
Circulation 2004;110:107-108.
FULL TEXT
The Classic Caries Clinical Trial: Constraints and Opportunities
Stamm
JDR 2004;83:C6-C14.
FULL TEXT
Clinical guidelines online: do they improve compliance?
Williams et al.
Postgrad. Med. J. 2004;80:415-419.
ABSTRACT
| FULL TEXT
CONSORT statement: extension to cluster randomised trials
Campbell et al.
BMJ 2004;328:702-708.
FULL TEXT
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt et al.
Fam Pract 2004;21:4-10.
ABSTRACT
| FULL TEXT
Standardized Reporting of Clinical Practice Guidelines: A Proposal from the Conference on Guideline Standardization
Shiffman et al.
ANN INTERN MED 2003;139:493-498.
ABSTRACT
| FULL TEXT
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Am. J. Roentgenol. 2003;181:51-55.
ABSTRACT
| FULL TEXT
Pitfalls in the design and analysis of efficacy trials in subfertility: Associate editor's commentary: on the article 'Common statistical errors in the design and analysis of subfertility trials' by A.Vail and E.Gardner.
Daya
Hum Reprod 2003;18:1005-1009.
FULL TEXT
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
ANN INTERN MED 2003;138:40-44.
ABSTRACT
| FULL TEXT
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
ANN INTERN MED 2003;138:W1-W12.
ABSTRACT
| FULL TEXT
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt et al.
BMJ 2003;326:41-44.
ABSTRACT
| FULL TEXT
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Clin. Chem. 2003;49:1-6.
ABSTRACT
| FULL TEXT
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
Clin. Chem. 2003;49:7-18.
ABSTRACT
| FULL TEXT
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Radiology 2003;226:24-28.
ABSTRACT
| FULL TEXT
Reporting Enrollment in Clinical Trials
Gross and Krumholz
ANN INTERN MED 2002;137:1008-1008.
FULL TEXT
Randomized Clinical Trial of an Implantable Drug Delivery System Compared With Comprehensive Medical Management for Refractory Cancer Pain: Impact on Pain, Drug-Related Toxicity, and Survival
Smith et al.
JCO 2002;20:4040-4049.
ABSTRACT
| FULL TEXT
Alefacept for Psoriasis: Promising Drug, Open Questions
Naldi
Arch Dermatol 2002;138:1238-1240.
FULL TEXT
Reporting the Recruitment Process in Clinical Trials: Who Are These Patients and How Did They Get There?
Gross et al.
ANN INTERN MED 2002;137:10-16.
ABSTRACT
| FULL TEXT
Reporting Number Needed to Treat and Absolute Risk Reduction in Randomized Controlled Trials
Nuovo et al.
JAMA 2002;287:2813-2814.
ABSTRACT
| FULL TEXT
Are randomised controlled trials in the BMJ different?
Egger et al.
BMJ 2001;323:1253-1253.
FULL TEXT
Systematic reviews in health care: Assessing the quality of controlled clinical trials
Juni et al.
BMJ 2001;323:42-46.
FULL TEXT
Better standards for better reporting of RCTs
Bossuyt
BMJ 2001;322:1317-1318.
FULL TEXT
The CONSORT Statement: Revised Recommendations for Improving the Quality of Reports of Parallel-Group Randomized Trials
Moher et al.
JAMA 2001;285:1987-1991.
ABSTRACT
| FULL TEXT
Use of the CONSORT Statement and Quality of Reports of Randomized Trials: A Comparative Before-and-After Evaluation
Moher et al.
JAMA 2001;285:1992-1995.
ABSTRACT
| FULL TEXT
CONSORT Revised--Improving the Reporting of Randomized Trials
Rennie
JAMA 2001;285:2006-2007.
FULL TEXT
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman et al.
ANN INTERN MED 2001;134:663-694.
ABSTRACT
| FULL TEXT
|
