 |
|
Eligibility Criteria of Randomized Controlled Trials Published in High-Impact General Medical JournalsA Systematic Sampling Review
Harriette G. C. Van Spall, MD;
Andrew Toren, MD;
Alex Kiss, PhD;
Robert A. Fowler, MD, MS
JAMA. 2007;297:1233-1240.
ABSTRACT
| |
Context Selective eligibility criteria of randomized controlled trials (RCTs) are vital to trial feasibility and internal validity. However, the exclusion of certain patient populations may lead to impaired generalizability of results.
Objective To determine the nature and extent of exclusion criteria among RCTs published in major medical journals and the contribution of exclusion criteria to the representation of certain patient populations.
Data Sources and Study Selection The MEDLINE database was searched for RCTs published between 1994 and 2006 in certain general medical journals with a high impact factor. Of 4827 articles, 283 were selected using a series technique.
Data Extraction Trial characteristics and the details regarding exclusions were extracted independently. All exclusion criteria were graded independently and in duplicate as either strongly justified, potentially justified, or poorly justified according to previously developed and pilot-tested guidelines.
Data Synthesis Common medical conditions formed the basis for exclusion in 81.3% of trials. Patients were excluded due to age in 72.1% of all trials (60.1% in pediatric populations and 38.5% in older adults). Individuals receiving commonly prescribed medications were excluded in 54.1% of trials. Conditions related to female sex were grounds for exclusion in 39.2% of trials. Of all exclusion criteria, only 47.2% were graded as strongly justified in the context of the specific RCT. Exclusion criteria were not reported in 12.0% of trials. Multivariable analyses revealed independent associations between the total number of exclusion criteria and drug intervention trials (risk ratio, 1.35; 95% confidence interval, 1.11-1.65; P = .003) and between the total number of exclusion criteria and multicenter trials (risk ratio, 1.26; 95% confidence interval, 1.06-1.52; P = .009). Industry-sponsored trials were more likely to exclude individuals due to concomitant medication use, medical comorbidities, and age. Drug intervention trials were more likely to exclude individuals due to concomitant medication use, medical comorbidities, female sex, and socioeconomic status. Among such trials, justification for exclusions related to concomitant medication use and comorbidities were more likely to be poorly justified.
Conclusions The RCTs published in major medical journals do not always clearly report exclusion criteria. Women, children, the elderly, and those with common medical conditions are frequently excluded from RCTs. Trials with multiple centers and those involving drug interventions are most likely to have extensive exclusions. Such exclusions may impair the generalizability of RCT results. These findings highlight a need for careful consideration and transparent reporting and justification of exclusion criteria in clinical trials.
INTRODUCTION
Randomized controlled trials (RCTs) are generally accepted as the most unbiased measures of efficacy for new interventions, drugs, or devices.1 Results of well-conducted RCTs provide clinicians and health policy makers with the best evidence for adoption or rejection of new therapies.2 The findings of RCTs published in major medical journals effect change in medical practice.3 Much attention is paid to the internal validity of clinical trials.4 However, even the most well-designed clinical trials are of limited use to clinicians if the results have poor external validity and are not generalizable to the patient population for whom the intervention may be applied.4
The external validity of clinical investigations may be threatened by limited patient access to trial centers, selection bias, lack of patient consent to enrollment, and physician resistance to randomization of patients due to treatment preferences5-10; however, rigorous inclusion and exclusion criteria may be the greatest challenge to generalizability of results.11-16 A recent study demonstrated that among 20 388 Medicare beneficiaries discharged from acute care hospitals in the United States with the principal diagnosis of heart failure, only 13% to 25% met the enrollment criteria of 3 landmark RCTs that have influenced the therapies of all patients with congestive heart failure.17
Appropriate eligibility criteria are essential for the design of internally valid and efficient RCTs that aim to measure efficacy of an intervention. However, several patient populations, women and children most prominently, have been historically underrepresented in clinical investigations.18-28 While it does not make biological sense to enroll women in a prostate cancer trial, it might be reasonable to allow enrollment of males in a breast cancer trial. However, the rationale underlying most exclusion criteria is much less clear. For example, enrollment of women in clinical cardiovascular investigations has been disproportionately smaller than the prevalence of disease among women in the general population.23 Within handbooks on drug safety and efficacy for children and pregnant and lactating women, the most common directive for common medications is that insufficient information is available within these populations.25, 29-32 Similar situations frequently exist for certain ethnic groups, the very elderly, and patients with chronic medical illnesses.17-22,29-30 Advocacy groups and national funding organizations have voiced concern regarding the exclusion of these patient populations from clinical trials.21, 33-39
The balance between establishing an intervention's efficacy, which may require greater exclusions and a circumscribed patient population, and effectiveness, which necessitates a less restrictive range of participants, is challenging. In the absence of RCT data on those patient populations, clinicians may be compelled to prescribe with trepidation, and gather knowledge about safety and efficacy through anecdotal reports collected outside the structure and safety of clinical trials. Without knowledge of effectiveness, clinicians may offer suboptimal treatment to patients who are underrepresented, or alternatively, expose them to unanticipated harms. The goal of this study was to determine the nature and extent of exclusion criteria among the RCTs published in major medical journals and the contribution of exclusion criteria to the underrepresentation of certain patient populations.
METHODS
Data Sources
We performed a structured sampling of articles identified through a systematic search of general medical journals with an impact factor of greater than 2.540-42 that published at a minimum of monthly intervals. Using text word or Medical Subject Headings clinical trial; randomized controlled trial; clinical trial, phase I; clinical trial, phase II; clinical trial, phase III; clinical trial, phase IV; and controlled clinical trial, the MEDLINE database was searched for relevant articles. The search was limited to RCTs involving human subjects published between January 1, 1994, and December 31, 2005, in the American Journal of Medicine, Annals of Internal Medicine, Archives of Internal Medicine, British Medical Journal, Canadian Medical Association Journal, Journal of the American Medical Association, Journal of General Internal Medicine, Lancet, and New England Journal of Medicine. More information about the journal search process is available from the authors and at http://www.sunnybrook.ca/research/scientists/AtoF/fowler.
Trial Selection
Our search strategy yielded 4827 articles (Figure). Based on findings of a pilot review involving 50 articles from 5 journals published over a 5-year period, we prospectively decided to sample 250 articles published over a 10-year period. Prior to publication, the sample was updated to include 283 articles over a 12-year period (January 1, 2004, to December 31, 2005). This decision was based on the accepted statistical practice of considering only 1 explanatory variable for every 10 outcome events observed in the sample.43 We selected every 16th article from our search, starting from the most recently published. This technique was used to prevent oversampling or undersampling relative to the total number of RCTs published within any 1 journal. Articles were excluded if they were not RCTs or if the trial primarily involved randomization and outcomes of participants or units other than individual patients (eg, physicians or clinics). Each excluded article was replaced with the next RCT published within the same journal, as identified by the MEDLINE search strategy. When identified trials presented long-term follow-up data from earlier reported trials in the selected journals, we confirmed trial characteristics with the original publication. Pilot, phase 1, and phase 2 RCTs are often not intended for immediate translation to patient care but instead are designed to find appropriate doses, ensure intervention safety, or aid with sample size calculations. Such trials were independently assessed by 2 of the authors (H.G.C.V. and R.A.F.) and generally excluded from the final analyses.
|
|
|
|
Figure. Flow Diagram of Screened, Eligible, and Included Randomized Controlled Trials
|
|
|
Data Extraction and Assessment of Methodological Quality
Three of the authors (H.G.C.V., A.T., and R.A.F.) independently reviewed all of the articles. The following variables were extracted from each article: publication date; source; corresponding author and country; sponsoring agencies; condition of interest; type of specialty and intervention (eg, drug, device, surgery, or other); university affiliation; single, multicenter, regional, national, or international enrollment; each inclusion and exclusion criterion and where they were listed; number of individuals assessed for eligibility, excluded, randomized, and followed up; and outcomes. When data were incomplete or unclear, we contacted the corresponding author for a complete list of eligibility criteria. The overall methodological quality of each trial was assessed using 4 parameters (generation of the allocation sequence, concealment of treatment allocation, blinding of outcome assessment, and handling of withdrawals and dropouts) that influence the ability of a trial to provide a true estimate of treatment effect.44-45
Definition of Inclusion and Exclusion Criteria
It was necessary to develop definitions that distinguished inclusion from exclusion criteria because ambiguity frequently existed (eg, inclusion criterion: patients with normal renal function; exclusion criterion: patients with renal insufficiency). Inclusion criteria were defined as criteria governing entry or recruitment of individuals into the trial and describing the medical condition of interest. All other criteria limiting the eligibility of individuals were treated as exclusion criteria. For example, among the following inclusion criteria listed in a trial of diabetic foot ulcers: "inpatient in a general medical ward," "age greater than or equal to 65 years," "diabetes," "grade 1 foot ulcers," and "normal renal function"; only "inpatient in a general medical ward," "diabetes," and "grade 1 foot ulcers" were treated as inclusion criteria; age and renal function were analyzed as exclusion criteria.
The reporting of each of the inclusion and exclusion criteria extracted from the articles was classified as strongly justified, potentially justified, and poorly justified. This classification was based on a search of the medical literature on optimal eligibility criteria. Key characteristics were grouped, reduced, and modified. Sensibility testing was performed among a group of clinicians and clinical investigators involved with both clinical trials and observational studies. The classification system was modified until the group was satisfied that it was able to categorize eligibility criteria appropriately and with high-interrater reliability. Exclusion criteria were deemed strongly justified if any specific rationale for the exclusion was provided, even when adjudicators would otherwise have classified the criterion as potentially or poorly justified. Criteria were pilot tested with 50 RCTs before final modification. The final classification criteria were applied without knowledge of other covariates and independently by 2 of the authors (H.G.C.V. and R.A.F.) and compared by the  statistic. Differences were decided by consensus or independent adjudication (Box).
| Box. Classification of Exclusion Criteria
Strongly Justified Reasons for Excluding Individuals From a Randomized Controlled Trial
- Individual or substitute decision maker is unable to grant informed consent
- Intervention or placebo would likely be harmful
- Unacceptable risk of known adverse reaction to intervention
- Unacceptable risk of assignment to placebo or withholding of intervention
- Intervention would likely be ineffective
- Individual not likely to have the condition of interest
- Individual not at risk for outcome
- Individual has type of disease that is likely not to respond to treatment
- Effect of intervention will be difficult to interpret
- Individual has a cointervention that will likely confound the treatment effect
- Individual has an independent condition with signs and symptoms similar to the condition of interest that will make the treatment effect difficult to assess
(eg, allergic rhinitis and upper respiratory tract infection)
Poorly Justified Reasons for Excluding Individuals From a Trial
- Is not a strongly justifiable reason as described above
- All of the following are true:
- The exclusion is based on
 1 of the following factors:
- Age
- Sex
- Sex-specific conditions such as menstruation, pregnancy, or lactation
- Racial, ethnic, or religious background
- Spoken or written language ability
- Educational background
- Socioeconomic status
- Cognitive ability or IQ
- Physical ability or disability
- Chronic health condition
- The condition under investigation and/or the intervention is not specific to the factors described above
- The factors described above have no direct bearing on the condition, intervention, or results
Potentially Justified Reasons for Excluding Individuals From a Trial
- Is neither a strongly justified reason nor a poorly justified reason as described above
- Individual may not adhere to intervention
- Individual may not complete follow-up
RETURN TO TEXT |
|
Data Synthesis
Descriptive data are presented as counts and percentages for categorical variables and mean (SD) or median (interquartile range) for continuous variables. Univariate analyses consisted of 2-sided t test or Mann-Whitney U test for continuous variables and  2 test or Fisher exact test for categorical variables. A Poisson regression model was used to examine the associations between trial characteristics and the number of specific exclusion criteria with estimates displayed by means of risk ratios (RRs) and their associated 95% confidence intervals (CIs). Associations between trial characteristics and the percentage of poorly justified exclusion criteria were examined using a multivariable linear regression model. All statistical tests were 2-tailed. Factors were considered statistically significant at a significance level of P<.05. Due to the inherent heterogeneity among the RCTs selected, quantitative meta-analytic techniques were not appropriate. For all analyses, SAS version 9.1 (SAS Institute Inc, Cary, NC) was used.
RESULTS
The sampled trials represented a variety of medical and surgical subspecialties, sponsorship, academic environments, and scope (Table 1). Evaluation of a medication was the single most common trial focus. Whether the trial was of a medical or surgical patient population, the most common conditions of focus for the trial included infectious disease, cardiovascular disease, cancer, and neurological disorder.
|
|
|
|
Table 1. Trial Characteristics
|
|
|
Exclusion criteria were reported in 88.0% of the RCTs, most commonly under the methods section but occasionally within the results section or an appendix. Of the RCTs that did not report exclusion criteria within the text of the publication, only 50% directed the reader to a reference article for exclusion criteria. Age was a reason for exclusion in 72.1% of the RCTs (Table 2). Potential participants younger than 16 years were excluded in 60.1% of trials; those older than 65 years in 38.5% of the RCTs. Restrictions in the inclusion of older adults applied to individuals ranging from age 50 to 80 years.
|
|
|
|
Table 2. Categories of Exclusion Criteria
|
|
|
Conditions related to sex were grounds for exclusion in 47.0% of all trials (Table 2). While females were specifically excluded from 6.7% of trials, conditions associated with female sex were excluded from 39.2% of the RCTs. These included pregnancy (31.9%), lactation (14.5%), lack of contraception use (8.8%), and menopausal status (3.9%). Male sex was the reason for exclusion in only 7.8% of the RCTs.
Individuals were excluded from trial participation due to medical comorbidities in 81.3% of the RCTs, most commonly for kidney, infectious, cardiac, liver, or hematological-oncological disease (Table 2). Many trials (30.9%) did not specify the medical condition that warranted exclusion but listed terms such as "illness," "comorbidity," or "significant disease." Medication-related reasons were the cause of exclusion in 54.1% of trials. Individuals with human immunodeficiency virus or AIDS were excluded from 8.9% of trials, and 12.1% of the RCTs excluded patients on the basis of limited life expectancy. Other common reasons for exclusion included socioeconomic status (13.8%), communication or language barriers (10.6%), participation in other trials (7.1%), and ethnicity (2.1%).
Application of the justification framework by the authors (Box) to the individual exclusion criteria revealed strong agreement (weighted , 0.80; 95% CI, 0.78-0.82). A majority (84.1%) of trials contained at least 1 poorly justified exclusion criterion; one quarter of all exclusions were poorly justified in 61.5% of the RCTs (Table 3). Of the 2709 individual exclusion criteria identified, only 47.2% were deemed strongly justified in the context of the individual RCT, 15.9% were graded as potentially justified, and 37.1% as poorly justified. The most common categories of poorly justified participant exclusions were age, medical comorbidity, sex, medication use, and socioeconomic status (Table 3).
|
|
|
|
Table 3. Justification of Exclusion Criteria
|
|
|
Multivariable analyses revealed independent associations between the total number of exclusion criteria and drug intervention trials (RR, 1.35; 95% CI, 1.11-1.65; P = .003) and between the total number of exclusion criteria and multicenter trials (RR, 1.26; 95% CI, 1.06-1.52; P = .009) (Table 4). There was an independent association between the proportion of poorly justified exclusion criteria and trials focused on a medical (vs nonmedical) condition (coefficient estimate, 14.5%; 95% CI, 6.1%-22.9%; P = .001) (Table 5).
|
|
|
|
Table 4. Independent Associations Among Trial Characteristics and Number of Exclusion Criteria
|
|
|
|
|
|
|
Table 5. Independent Associations Among Trial Characteristics and Percentage of Poorly Justified Exclusion Criteria
|
|
|
Trials with drug interventions (compared with nondrug interventions) were more likely to exclude individuals due to concomitant medication use, medical comorbidities, female sex, and disadvantaged socioeconomic status. Exclusions due to concomitant medication use and medical comorbidities were more likely to be poorly justified (Table 6). Trials of an industry or pharmaceutical-sponsored intervention were more likely to exclude individuals due to concomitant medication use, medical comorbidities, and age. Exclusions due to concomitant medication use and medical comorbidities were more likely to be poorly justified (Table 7).
|
|
|
|
Table 6. Associations Among Exclusion Criteria and Drug Intervention Trials
|
|
|
|
|
|
|
Table 7. Associations Among Exclusion Criteria and Industry-Sponsored Trials
|
|
|
To investigate whether there was a change in eligibility criteria over time, the RCTs were divided into 2 periods (1994-1998 and 1999-2005), which roughly corresponded to the original publication and dissemination of the CONSORT (Consolidated Standards of Reporting Trials) statements between 1996 and 1998. Compared with the earlier period, the RCTs published between 1999 and 2005 were associated with a nonsignificant improvement in the trial quality score (P = .07), a nonsignificant increase in the mean number of exclusion criteria per trial (8.7 for 1994-1998 vs 10.1 for 1999-2005; P = .05), and a significant reduction in the percentage of poorly justified trial criteria (36.6% for 1994-1998 vs 26.5% for 1999-2005; P = .003).
COMMENT
In this review, we found that the RCTs published in major medical journals may exclude both large segments of the general population and also specific patient populations from the benefits of participation in clinical investigations. We found frequent exclusions of children, the elderly, women (particularly those lactating, pregnant, or able to become pregnant), patients taking common medications, and those with common medical comorbidities. Reasons for exclusions were frequently not justified in the context of individual RCTs. The RCTs with a focus on multicenter drug evaluation were more likely to have a greater number of exclusion criteria. Drug intervention trials and trials with industry sponsorship were associated with exclusions related to comorbidity, medication use, and occasionally age, sex, and socioeconomic status, often without adequate justification.
There are benefits of stringent eligibility criteria. The inclusion and exclusion criteria in a RCT are designed to identify a population of interest in whom an intervention has the greatest likelihood to produce a clinically important and statistically significant effect.38 Efficacy trials with well-defined and homogeneous populations can generally be smaller, shorter, more efficient, and less expensive. This is desirable because clinical trials are increasingly challenged by high costs, limited funding, and regulatory restrictions.39 However, there may be a reciprocal eligibility relationship between efficacy and effectiveness with an inevitable loss of external validity in any tightly designed RCT. The advantages of stringent eligibility criteria are achieved at the risk of excluding patients who may be more likely to represent the population treated in clinical settings and who would better test an intervention's effectiveness.
It is likely that pregnant and lactating women, children, the elderly, and those with medical comorbidities are often excluded from clinical trials out of concern for their safety or the safety of a fetus. However, there is evidence that trial participants assigned to either intervention or placebo have fewer undesirable clinical events and lower mortality rates than those of eligible nonparticipants.46-48 Exclusions may also leave future patients with similar characteristics susceptible to unintended harm from an inappropriate generalization of trial results. This was illustrated by the dramatic increase in the rates of hyperkalemia-associated morbidity and mortality following publication of the Randomized Aldactone Evaluation (RALES) trial, largely attributed to the application of trial results to populations that were excluded from the RCT.49 There has been similar long-standing prescribing uncertainty for blacks with hypertension due to limited RCT generalizability.50 The recent controversies around cyclooxygenase-2 inhibitors may in part stem from narrowly defined patient populations in the original RCTs.51
Children may be especially susceptible because adverse drug reactions represent a significant cause of pediatric mortality and morbidity.27-31 The National Institutes of Health and others have mandated the inclusion of children in all research, barring scientific or ethical reasons.52-53 Although initiatives such as the Pediatric Exclusivity Program have improved the translation of knowledge of appropriate and safe prescribing practices to children, our results and those of others indicate that there is still need for careful consideration of age-based exclusions.54
The National Institutes of Health has similarly stipulated that research protocols must provide appropriate justification for the exclusion of women from trials and that women should be included in every clinical trial studying a disorder that affects women, irrespective of cost considerations.33-36 However, these initiatives have produced only minor changes in the sex composition of cohorts in many areas of clinical investigation.20, 37, 55 Underrepresentation of women in clinical trials relative to disease prevalence may partially be due to eligibility criteria that exclude individuals on the basis of lactation, pregnancy, or the potential to become pregnant.
The decision to enroll potentially pregnant or lactating women into RCTs should involve careful consideration of specific risks and benefits of the intervention. Still, many drug and nondrug interventions could be safely studied in this population; inclusion of women in these trials could allow the safety and efficacy to be determined prospectively in a highly monitored setting and could afford increased generalizability to those at risk. Providing safe and effective treatments to pregnant women would promote better outcomes for infants, as recently demonstrated in a pregnancy trial for zidovudine sponsored by the National Institutes of Health, which was terminated early due to a dramatic reduction in perinatal human immunodeficiency virus transmission in the group treated with zidovudine.56
The strengths of our study include the volume of RCTs systematically and independently reviewed, which were published in high-impact medical journals between 1994 and 2006. This minimizes the likelihood that our findings are due to selected sampling, chance, or single reviewer bias. Our methods, sample size, and system for justification were refined through an iterative process of pilot testing and modification among clinicians and clinical investigators with expertise in both observational and experimental designs. There was good agreement between reviewers. Finally, we believe our results are compelling.
Limitations of our study should be noted. We chose to focus on exclusion criteria, although there are many factors that may affect the external validity of RCTs, and these may deserve similar attention.57 We did not examine journals with a focus on subspecialty populations (for example, pediatrics or obstetrics). However, our purpose was not to examine subspecialty journals that report clinical trials of interventions in specific and narrowly defined populations. Instead, we examined the world's most widely read and quoted general medical journals that publish trials pertaining to conditions of broad interest to physicians and of potential relevance to broad groups of patients. Analysis of trials published in subspecialty journals may have yielded different results. Because we found great variability in the reporting of exclusion criteria among our cohort of RCTs, there may have been underreporting of some exclusion criteria, and our results may represent an underestimation of the true extent of exclusions. Another limitation is that only a minority of trials reported complete participant flow, making it difficult to establish the number of patients eliminated from trial participation on the basis of exclusion criteria alone. However, we attempted to mitigate this by contacting individual authors to obtain such information. Finally, we acknowledge that exclusion criteria are an essential and valuable methodological component of RCT design; however, we have examined the justification of such criteria to determine whether some patients may be unnecessarily excluded from participation. While such a classification is inherently subjective, we have used a prospective and transparent framework on which to base our grading, and independently examined each criterion in duplicate, with a high degree of agreement between the authors.
A perfect balance between efficacy and effectiveness is difficult to achieve within 1 RCT. Distinct RCTs to assess effectiveness might naturally follow preliminary efficacy trials. However, if an intervention has been proven efficacious within an internally valid RCT, it is practically difficult to undertake effectiveness trials because clinical equipoise may no longer exist; unless mandated by a regulatory body, there is limited incentive for industry to fund a RCT when a more modest effect may be expected; and, other agencies rarely have resources to support such a follow-up RCT. Efficacy and effectiveness trials might proceed concurrently within a single large RCT or as separate trials; however, this will also inflate the sample size and cost. The safest and most efficient manner to test interventions may be the current schedule of preclinical work, followed by dose-finding, safety and efficacy trials, with subsequent registries and surveillance systems. Mandatory multiyear registries for patients subsequently treated but who would have been excluded from the original trial is an additional mechanism; yet, this may deter treatment use and be an inferior substitute for sound clinical assessment of the risks and benefits of any therapy.
Based on the results of this study, we recommend that every reasonable effort be made to minimize exclusions of specific patient populations when such patients will likely form a group to which the results are generalized. The CONSORT guidelines and the Uniform Requirements for Manuscripts Submitted to Biomedical Journals have been major steps forward in improving the transparency of RCTs,45, 58 and we did find evidence of fewer unjustified criteria over time; however, further improvements in adherence and reporting can be made.57 When exclusions are necessary, we recommend that each criterion be clearly justified within the methods section or appendix. Nonspecific terms such as "medical illness" or "renal insufficiency" should be avoided, opting for more objective criteria (eg, "patients with estimated glomerular filtration rate <30 mL/min"). We recommend that trial inclusion and exclusion criteria be clearly listed within a table that is accessible and apparent to the reader.57 This could take the form of a table entitled "who was included and excluded from this trial." This would not only enable clinicians to more clearly evaluate the generalizability of the results to other patients but also serve as a reminder of the need to consider all groups when designing and reporting clinical trials. Investigators must continue to strive toward making the fruits of clinical research available and relevant to as much of the population as possible: children and adults, females and males, and the sick as well as the healthy.
AUTHOR INFORMATION
Corresponding Author: Robert A. Fowler, MD, MS, Sunnybrook Hospital, Room D478, 2075 Bayview Ave, Toronto, Ontario, Canada M5N 3M5 (rob.fowler{at}sunnybrook.ca).
Author Contributions: Dr Fowler had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Van Spall, Fowler.
Acquisition of data: Van Spall, Toren, Fowler.
Analysis and interpretation of data: Van Spall, Kiss, Fowler.
Drafting of the manuscript: Van Spall, Kiss, Fowler.
Critical revision of the manuscript for important intellectual content: Van Spall, Fowler.
Statistical analysis: Kiss, Fowler.
Obtained funding: Fowler.
Administrative, technical, or material support: Fowler.
Study supervision: Fowler.
Financial Disclosures: None reported.
Funding/Support: Dr Fowler is a Career Scientist of the Ontario Ministry of Health and Long-term Care.
Role of the Sponsor: The sponsor had no role in the design; conduct of the study; collection, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
Acknowledgment: We thank the following people for their thoughtful review and suggestions on the methods and presentation of this work, none of whom were compensated: Steven Shumak, MD, Don Redelmeier, MD, MS, Neill Adhikari, MD, MSc, Damon Scales, MD, Ruxandra Pinto, PhD, Art Slutsky, MD, William Sibbald, MD, MPH (University of Toronto, Toronto, Ontario); Deborah Cook, MD, MSc (McMaster University, Hamilton, Ontario); Eva Shimaoka, MD (Toronto, Ontario); Daren Heyland, MD, MSc (Queen's University, Kingston, Ontario); and Gordon Rubenfeld, MD, MSc (University of Washington, Seattle). We dedicate this article to the memory of Aubrey Ignatius Van Spall.
Author Affiliations: Department of Medicine (Drs Van Spall and Fowler), Division of Cardiology (Dr Van Spall), Interdepartmental Division of Critical Care Medicine (Dr Fowler), Sunnybrook Health Sciences Centre (Drs Kiss and Fowler), University of Toronto, Toronto, Ontario; and Department of Ophthalmology, University of Ottawa, Ottawa, Ontario (Dr Toren).
REFERENCES
| |
1. Guyatt GH, Haynes RB, Jaeschke RZ, et al, Evidence-Based Medicine Working Group. Users' guides to the medical literature, XXV: evidence-based medicine: principles for applying the users' guides to patient care. JAMA. 2000;284:1290-1296.
FREE FULL TEXT
2. National Guidelines Clearinghouse Organizations. http://www.guidelines.gov. Accessed November 22, 2003.3. Guyatt GH, Sinclair J, Cook DJ, Glasziou P, Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. Users' guides to the medical literature, XVI: how to use a treatment recommendation. JAMA. 1999;281:1836-1843.
FREE FULL TEXT
4. Guyatt GH, Sackett DL, Cook DJ, Evidence-Based Medicine Working Group. Users' guides to the medical literature, II: how to use an article about therapy or prevention; A: are the results of the study valid? JAMA. 1993;270:2598-2601.
FREE FULL TEXT
5. Guyatt GH, Sackett DL, Cook DJ, Evidence-Based Medicine Working Group. Users' guides to the medical literature, II: how to use an article about therapy or prevention; B: what were the results and will they help me in caring for my patients? JAMA. 1994;271:59-63.
FREE FULL TEXT
6. Schain WS. Barriers to clinical trials, part II: knowledge and attitudes of potential participants. Cancer. 1994;74(suppl):2666-2671.
FULL TEXT
|
ISI
| PUBMED
7. Cassileth BR, Lusk EJ, Miller DS, Hurwitz S. Attitudes toward clinical trials among patients and the public. JAMA. 1982;248:968-970.
FREE FULL TEXT
8. Taylor KM, Margolese RG, Soskolne CL. Physicians' reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer. N Engl J Med. 1984;310:1363-1367.
ABSTRACT
9. Mansour EG. Barriers to clinical trials, part III: knowledge and attitudes of health care providers. Cancer. 1994;74(suppl):2672-2675.
FULL TEXT
|
ISI
| PUBMED
10. Corrie P, Shaw J, Harris R. Rate limiting factors in recruitment of patients to clinical trials in cancer research: descriptive study. BMJ. 2003;327:320-321.
FREE FULL TEXT
11. Fossa SD, Skovlund E. Selection of patients may limit the generalizability of results from cancer trials. Acta Oncol. 2002;41:131-137.
FULL TEXT
|
ISI
| PUBMED
12. Bijker N, Peterse JL, Fentiman IS, et al. Effects of patient selection on the applicability of results from a randomised clinical trial (EORTC 10853) investigating breast-conserving therapy for DCIS. Br J Cancer. 2002;87:615-620.
FULL TEXT
|
ISI
| PUBMED
13. Sokka T, Pincus T. Most patients receiving routine care for rheumatoid arthritis in 2001 did not meet inclusion criteria for most recent clinical trials or American college of rheumatology criteria for remission. J Rheumatol. 2003;30:1138-1146.
FREE FULL TEXT
14. Schneider LS, Olin JT, Lyness SA, Chui HC. Eligibility of Alzheimer's disease clinic patients for clinical trials. J Am Geriatr Soc. 1997;45:923-928.
ISI
| PUBMED
15. Fuks A, Weijer C, Freedman B, Shapiro S, Skrutkowska M, Riaz A, National Surgical Adjuvant Breast and Bowel Program and the Pediatric Oncology Group. A study in contrasts: eligibility criteria in a twenty-year sample of NSABP and POG clinical trials. J Clin Epidemiol. 1998;51:69-79.
FULL TEXT
|
ISI
| PUBMED
16. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy. 1999;4:112-121.
PUBMED
17. Masoudi FA, Havranek EP, Wolfe P, et al. Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. Am Heart J. 2003;146:250-257.
FULL TEXT
|
ISI
| PUBMED
18. Heiat A, Gross CP, Krumholz HM. Representation of the elderly, women, and minorities in heart failure clinical trials. Arch Intern Med. 2002;162:1682-1688.
FREE FULL TEXT
19. Rathore SS, Krumholz HM. Race, ethnic group, and clinical research. BMJ. 2003;327:763-764.
FREE FULL TEXT
20. Lee PY, Alexander KP, Hammill BG, Pasquali SK, Peterson ED. Representation of elderly persons and women in published randomized trials of acute coronary syndromes. JAMA. 2001;286:708-713.
FREE FULL TEXT
21. Hall WD. Representation of blacks, women, and the very elderly (aged > or = 80) in 28 major randomized clinical trials. Ethn Dis. 1999;9:333-340.
PUBMED
22. Bartlett C, Davey P, Dieppe P, Doyal L, Ebrahim S, Egger M. Women, older persons, and ethnic minorities: factors associated with their inclusion in randomised trials of statins 1990 to 2001. Heart. 2003;89:327-328.
FREE FULL TEXT
23. Douglas PS. Gender, cardiology, and optimal medical care. Circulation. 1986;74:917-919.
FREE FULL TEXT
24. Blumer JL. Off-label uses of drugs in children. Pediatrics. 1999;104:598-602.
FREE FULL TEXT
25. US Food and Drug Administration. Pediatric Exclusivity Provision: January 2001 Status Report to Congress. Rockville, Md: Food and Drug Administration; 2001.26. Department of Health and Human Services; Food and Drug Administration. Specific requirements on content and format of labeling for human prescription drugs: revision of "pediatric use" subsection in the labeling: final rule (21 CFR part 201). Fed Regist. 1994;59:64240-64241.
27. Steinbrook R. Testing medications in children. N Engl J Med. 2002;347:1462-1470.
FREE FULL TEXT
28. Matsui D, Kwan C, Steer E, Rieder MJ. The trials and tribulations of doing drug research in children. CMAJ. 2003;169:1033-1034.
FREE FULL TEXT
29. Medco Health Solutions. Drug Trend Report. Franklin Lakes, NJ: Medco Health Solutions; 2002.30. Wilson JT. An update on the therapeutic orphan. Pediatrics. 1999;104:585-590.
FREE FULL TEXT
31. Moore TJ, Weiss SR, Kaplan S, Blaisdell CJ. Reported adverse drug events in infants and children under 2 years of age. Pediatrics. 2002;110:e53.
FREE FULL TEXT
32. Briggs GG, Freeman RK, Yaffe SJ. Drugs in Pregnancy and Lactation. Baltimore, Md: Williams & Wilkins; 1998.33. National Institutes of Health. NIH Guide for Grants and Contracts. Bethesda, Md: National Institutes of Health; 1987.34. National Institutes of Health. NIH Guide for Grants and Contracts. Bethesda, Md: National Institutes of Health; 1990.35. National Institutes of Health Revitalization Act of 1993, Pub L No. 103-43 (1993).36. National Institutes of Health. NIH guidelines on the inclusion of women and minorities as subjects in clinical research. Fed Regist. 1994;59:14508-14513.
37. Harris DJ, Douglas PS. Enrollment of women in cardiovascular clinical trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med. 2000;343:475-480.
FREE FULL TEXT
38. Hulley SB, Newman TB, Cummings SR. Choosing the study subjects: specification, sampling, and recruitment. In: Hulley SB, Cummings SR, Browner WS, et al, eds. Designing Clinical Research. 2nd ed. Philadelphia, Pa: Lippincott Williams & Wilkins; 2001:25-34.39. Sung NS, Crowley WF Jr, Genel M, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289:1278-1287.
FREE FULL TEXT
40. Egger M, ed, Davey Smith G, ed, Altman D, ed. Systematic Reviews in Health Care: Meta-Analysis in Context. 2nd ed. London, England: BMJ Books; 2001:429-443.41. Garfield E. Which medical journals have the greatest impact? Ann Intern Med. 1986;105:313-320.
FREE FULL TEXT
42. Thomson Corporation. ISI Web of Knowledge. http://isiwebofknowledge.com. Accessibility verified February 26, 2007.43. Harrell FE Jr, Lee KL, Matchar DB, Reichert TA. Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep. 1985;69:1071-1077.
ISI
| PUBMED
44. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analyses. JAMA. 1999;282:1054-1060.
FREE FULL TEXT
45. Moher D, Schulz KF, Altman D, CONSORT (Consolidated Standards of Reporting Trials) Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285:1987-1991.
FREE FULL TEXT
46. Vist GE, Hagen KB, Devereaux PJ, Oxman AD. Outcomes of patients who participate in randomized controlled trials versus those of similar patients who do not participate [Cochrane Library on CD-ROM]. Oxford, England: Update Software; 2001; issue 4.47. Davis S, Wright PW, Schulman SF, et al. Participants in prospective, randomized clinical trials for resected non-small cell lung cancer have improved survival compared with nonparticipants in such trials. Cancer. 1985;56:1710-1718.
FULL TEXT
|
ISI
| PUBMED
48. Schmidt B, Gillie P, Caco C, Roberts J, Roberts R. Do sick newborns benefit from participation in a randomized clinical trial? J Pediatr. 1999;134:151-155.
ISI
| PUBMED
49. Juurlink DN, Mamdani MM, Lee DS, et al. Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study. N Engl J Med. 2004;351:543-551.
FREE FULL TEXT
50. Flack JM, Mensah GA, Ferrario CM. Using angiotensin converting enzyme inhibitors in African-American hypertensives: a new approach to treating hypertension and preventing target-organ damage. Curr Med Res Opin. 2000;16:66-79.
FULL TEXT
|
ISI
| PUBMED
51. Mamdani D, Juurlink D, Lee D, et al. Cyclo-oxygenase-2 inhibitors versus non-selective non-steroidal anti-inflammatory drugs and congestive heart failure outcomes in elderly patients: a population-based cohort study. Lancet. 2004;363:1751-1756.
FULL TEXT
|
ISI
| PUBMED
52. Caldwell PH, Murphy SB, Butow PN, Craig JC. Clinical trials in children. Lancet. 2004;364:803-811.
FULL TEXT
|
ISI
| PUBMED
53. National Institute of Health NIH policy and guidelines on the inclusion of children as participants in research involving human subjects. http://grants.nih.gov/grants/guide/notice-files/not98-024.html. Accessed July 31, 2006.54. Benjamin DK Jr, Smith PB, Murphy MD, et al. Peer-reviewed publication of clinical trials completed for pediatric exclusivity. JAMA. 2006;296:1266-1273.
FREE FULL TEXT
55. US Department of Health and Human Services; US Food and Drug Administration. Executive summary: FDA Regulated Products and Pregnant Women. Washington, DC: Food and Drug Administration; 1994.56. Connor EM, Sperling RS, Gelber R, et al, Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. N Engl J Med. 1994;331:1173-1180.
FREE FULL TEXT
57. Rothwell PM. External validity of randomised controlled trials: to whom do the results of this trial apply? Lancet. 2005;365:82-93.
FULL TEXT
|
ISI
| PUBMED
58. International Committee of Medical Journal Editors Web site Uniform requirements for manuscripts submitted to biomedical journals. http://www.icmje.org. Accessed January 27, 2006.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
RELATED LETTERS
Eligibility Criteria of Randomized Controlled Trials
Giorgio Costantino, Elisa Ceriani, Anna Maria Rusconi, and Nicola Montano
JAMA. 2007;298(1):39.
EXTRACT
| FULL TEXT
Eligibility Criteria of Randomized Controlled Trials—Reply
Robert A. Fowler and Harriette G. C. Van Spall
JAMA. 2007;298(1):39-40.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Impact of acute and chronic risk factors on use of evidence-based treatments in patients in Australia with acute coronary syndromes
Joynt et al.
Heart 2009;95:1442-1448.
ABSTRACT
| FULL TEXT
What kind of randomised trials do patients and clinicians need?
Zwarenstein and Treweek
Evid. Based Med. 2009;14:101-103.
FULL TEXT
Evaluation of the management of heart failure in primary care
Calvert et al.
Fam Pract 2009;26:145-153.
ABSTRACT
| FULL TEXT
Status of Women in Cardiovascular Clinical Trials
Kim and Menon
Arterioscler. Thromb. Vasc. Bio. 2009;29:279-283.
FULL TEXT
Has Mortality from Acute Respiratory Distress Syndrome Decreased over Time?: A Systematic Review
Phua et al.
Am. J. Respir. Crit. Care Med. 2009;179:220-227.
ABSTRACT
| FULL TEXT
Improving the reporting of pragmatic trials: an extension of the CONSORT statement
Zwarenstein et al.
BMJ 2008;337:a2390-a2390.
ABSTRACT
| FULL TEXT
Prophylaxis Against Deep Vein Thrombosis in Critically Ill Patients With Severe Renal Insufficiency With the Low-Molecular-Weight Heparin Dalteparin: An Assessment of Safety and Pharmacodynamics: The DIRECT Study
Douketis et al.
Arch Intern Med 2008;168:1805-1812.
ABSTRACT
| FULL TEXT
Varenicline versus transdermal nicotine patch for smoking cessation: results from a randomised open-label trial
Aubin et al.
Thorax 2008;63:717-724.
ABSTRACT
| FULL TEXT
Update on pharmacologic and nonpharmacologic therapies for smoking cessation
Schmelzle et al.
cfp 2008;54:994-999.
ABSTRACT
| FULL TEXT
Eligibility Criteria of Randomized Controlled Trials
Costantino et al.
JAMA 2007;298:39-39.
FULL TEXT
|
