 |
|
Effects of Quality Improvement Strategies for Type 2 Diabetes on Glycemic Control
A Meta-Regression Analysis
Kaveh G. Shojania, MD;
Sumant R. Ranji, MD;
Kathryn M. McDonald, MM;
Jeremy M. Grimshaw, MBChB, PhD;
Vandana Sundaram, MPH;
Robert J. Rushakoff, MD;
Douglas K. Owens, MD, MS
JAMA. 2006;296:427-440.
ABSTRACT
| |
Context There have been numerous reports of interventions designed to improve the care of patients with diabetes, but the effectiveness of such interventions is unclear.
Objective To assess the impact on glycemic control of 11 distinct strategies for quality improvement (QI) in adults with type 2 diabetes.
Data Sources and Study Selection MEDLINE (1966-April 2006) and the Cochrane Collaboration's Effective Practice and Organisation of Care Group database, which covers multiple bibliographic databases. Eligible studies included randomized or quasi-randomized controlled trials and controlled before-after studies that evaluated a QI intervention targeting some aspect of clinician behavior or organizational change and reported changes in glycosylated hemoglobin (HbA1c) values.
Data Extraction Postintervention difference in HbA1c values were estimated using a meta-regression model that included baseline glycemic control and other key intervention and study features as predictors.
Data Synthesis Fifty randomized controlled trials, 3 quasi-randomized trials, and 13 controlled before-after trials met all inclusion criteria. Across these 66 trials, interventions reduced HbA1c values by a mean of 0.42% (95% confidence interval [CI], 0.29%-0.54%) over a median of 13 months of follow-up. Trials with fewer patients than the median for all included trials reported significantly greater effects than did larger trials (0.61% vs 0.27%, P = .004), strongly suggesting publication bias. Trials with mean baseline HbA1c values of 8.0% or greater also reported significantly larger effects (0.54% vs 0.20%, P = .005). Adjusting for these effects, 2 of the 11 categories of QI strategies were associated with reductions in HbA1c values of at least 0.50%: team changes (0.67%; 95% CI, 0.43%-0.91%; n = 26 trials) and case management (0.52%; 95% CI, 0.31%-0.73%; n = 26 trials); these also represented the only 2 strategies conferring significant incremental reductions in HbA1c values. Interventions involving team changes reduced values by 0.33% more (95% CI, 0.12%-0.54%; P = .004) than those without this strategy, and those involving case management reduced values by 0.22% more (95% CI, 0.00%-0.44%; P = .04) than those without case management. Interventions in which nurse or pharmacist case managers could make medication adjustments without awaiting physician authorization reduced values by 0.80% (95% CI, 0.51%-1.10%), vs only 0.32% (95% CI, 0.14%-0.49%) for all other interventions (P = .002).
Conclusions Most QI strategies produced small to modest improvements in glycemic control. Team changes and case management showed more robust improvements, especially for interventions in which case managers could adjust medications without awaiting physician approval. Estimates of the effectiveness of other specific QI strategies may have been limited by difficulty in classifying complex interventions, insufficient numbers of studies, and publication bias.
INTRODUCTION
Diabetes mellitus has reached epidemic proportions in the United States.1-2 Despite well-established processes of care to reduce morbidity associated with diabetes, widespread quality problems exist.3-5 The literature contains numerous reports of interventions designed to remedy these problems, but their effectiveness remains unclear.6
Previous systematic reviews of quality improvement (QI) strategies for diabetes have provided only qualitative analysis7 or have focused on single types of interventions,8-15 leaving the relative effectiveness of different strategies unknown. This is particularly problematic, since many interventions include components of multiple QI strategies. Thus, some of the same trials16-21 may appear in reviews of patient education,13 disease management,11 and the impact of information technology on diabetes care.8 In each such review, the apparent impact of the QI strategy under consideration could be confounded by the presence of one or more other strategies in many of the trials.
The present review addresses these limitations by assessing the impact on glycemic control of 11 different categories of QI strategies to identify those that significantly augment intervention effectiveness. We began this work as part of a series of evidence reports on QI,6, 22 but this analysis includes roughly twice the number of trials as did our initial report, reflecting a marked surge in published evaluations of interventions to improve outpatient care of diabetes in recent years.
METHODS
Included Interventions
We included any study involving adult outpatients with type 2 diabetes in which the intervention met the definition for at least 1 of 11 specific types of QI strategies, based on a taxonomy adapted from the Cochrane Effective Practice and Organisation of Care (EPOC) group (Box).6, 23 As with other systematic reviews of the QI literature,24-28 these classifications are broad but pragmatic, reflecting the lack of descriptive detail typically provided in published evaluations of QI interventions.29-30 We included interventions involving patient education, self-management, or patient reminders only if the interventions involved at least 1 component directed at clinician behavior or organizational change, because multiple systematic reviews have already evaluated interventions that consist exclusively of patient-oriented strategies to improve diabetes care.10, 13-15,31
| Box. Taxonomy Used to Classify Quality Improvement Strategies
Audit and Feedback. Summary of clinical performance of health care delivered by an individual clinician or clinic over a specified period, which is then transmitted back to the clinician (eg, the percentage of a clinician's patients who have achieved a target glycosylated hemoglobin [HbA1c] level, or who have undergone a dilated-eye examination with a specified frequency).
Case Management. Any system for coordinating diagnosis, treatment, or ongoing patient management (eg, arrangement for referrals, follow-up of test results) by a person or multidisciplinary team in collaboration with or supplementary to the primary care clinician.
Team Changes. Changes to the structure or organization of the primary health care team, defined as present if any of the following applied:
• Adding a team member or "shared care," eg, routine visits with personnel other than the primary physician (including physician or nurse specialists in diabetic care, pharmacists, nutritionists, podiatrists).
• Use of multidisciplinary teams, ie, active participation of professionals from more than 1 discipline (eg, medicine, nursing, pharmacy, nutrition) in the primary, ongoing management of patients.
• Expansion or revision of professional roles (eg, nurse or pharmacist plays more active role in patient monitoring or adjusting medication regimens).
Electronic Patient Registry. General electronic medical record system or electronic tracking system for patients with diabetes.
Clinician Education. Interventions designed to promote increased understanding of principles guiding clinical care or awareness of specific recommendations for a target condition or patient population. Subcategories of clinician education included conferences or workshops, distribution of educational materials, and educational outreach visits.
Clinician Reminders. Paper-based or electronic system intended to prompt a health professional to recall patient-specific information (eg, most recent HbA1c value) or to perform a specific task (eg, perform a foot examination). If accompanied by a recommendation, the strategy would be subclassified as decision support.
Facilitated Relay of Clinical Information to Clinicians. Clinical information collected from patients and transmitted to clinicians by means other than the existing medical record. Conventional means of correspondence between clinicians were excluded. For example, if the results of routine visits with a pharmacist were sent in a letter to the primary care physician, the use of routine visits with a pharmacist would count as a "team" change, but the intervention would not also be counted as "facilitated relay." If, however, the pharmacist issues structured diaries for patients to record self-monitored glucose values, which are then brought in person to office visits to review with the primary physician, then the intervention would count as "facilitated relay." Other examples include electronic or Web-based tools through which patients provide self-care data and which clinicians review,23-25 as well as point-of-care testing supplying clinicians with immediate HbA1c values.26
Patient Education. Interventions designed to promote increased understanding of a target condition or to teach specific prevention or treatment strategies, or specific in-person patient education (eg, individual or group sessions with diabetes nurse educator; distribution of printed or electronic educational materials). Interventions with patient education were included only if they also included at least 1 other strategy related to clinician or organizational change.
Promotion of Self-Management. Provision of equipment (eg, home glucometers) or access to resources (eg, system for electronically transmitting home glucose measurements and receiving insulin dose changes based on those data) to promote self-management. Interventions promoting patient self-management were included only if they also included at least 1 other strategy related to clinician or organizational change.
Patient Reminder Systems. Any effort (eg, postcards or telephone calls) to remind patients about upcoming appointments or important aspects of self-care. Interventions with patient reminders were included only if they also included at least 1 other strategy related to clinician or organizational change.
Continuous Quality Improvement. Interventions explicitly identified as using the techniques of continuous quality improvement, total quality management, or plan-do-study-act, or any iterative process for assessing quality problems, developing solutions to those problems, testing their impacts, and then reassessing the need for further action.
Definitions of the 11 categories of quality improvement strategies were modified from the taxonomy used by the Cochrane Effective Practice and Organisation of Care (EPOC) group.6, 27 The definition for "facilitated relay of clinical information" corresponds to the EPOC category "patient mediated interventions"; we chose a different name to avoid confusion with patient education, self-management, and patient reminders. A category for financial incentives was originally included, but no studies involving this strategy met our inclusion criteria.
RETURN TO TEXT |
|
Included Study Designs and Outcomes
We included randomized controlled trials, quasi-randomized trials (eg, even- or odd-numbered medical records), and controlled before-after studies (eg, 2 similar clinics compared before and after implementation of an intervention at one of them). We restricted our analysis to studies reporting mean preintervention and postintervention glycosylated hemoglobin (HbA1c) values for each study group. Studies that reported changes in the percentage of patients who achieved a certain level of glycemic control varied too widely in terms of the target ranges to permit data pooling. However, for studies that met all other inclusion criteria, we contacted the authors to obtain data on mean HbA1c values and included the studies for which we received data.
Literature Search and Review Process
We combined Medical Subject Headings and text words related to QI with those related to diabetes to search the MEDLINE database up to April 2006 (search strategy available on request from the authors). We also retrieved all articles related to diabetes care from the Cochrane EPOC database, which is compiled on the basis of periodic extensive searches of MEDLINE, EMBASE, and CINAHL,32 and scanned bibliographies from key articles. Two investigators independently screened citations and abstracted included articles using a structured data entry form. Disagreements were resolved by consensus among 3 reviewers.
Analysis
We used a mixed model (Proc Mixed, SAS version 8.2; SAS Institute, Cary, NC) incorporating fixed and random effects to predict the difference in postintervention values for mean HbA1c values between intervention and control groups, based on baseline HbA1c values and other predictors of interest, such as QI strategies and key study features (eg, use of a randomized design). Postintervention standard deviations in the control and intervention groups were pooled, with weighting by sample size in each group. This measure of within-study variation provided the residual error in a mixed model with a random study effect.33-34
We anticipated that a substantial number of studies would allocate interventions to clinicians or clinics but collect data from patients (ie, the studies would exhibit "clustering"). To avoid spurious precision in patient-level outcomes,35-37 we calculated an effective sample size for each such trial: Neffective = (k x m)/(1 + (m – 1) x ICC), where k indicates the number of clusters; m, the number of observations per cluster; and ICC, the intracluster correlation coefficient.35, 38 We imputed unreported ICCs based on empirically derived values37 and varied these imputations in sensitivity analyses.
For quality assurance purposes, we independently reran the analyses in STATA version 8.2 (StataCorp, College Station, Tex) using the commands for meta-analysis and meta-regression.
RESULTS
Included Studies
The search yielded 5121 citations, 708 of which underwent full text review and 5817-21,39-94 of which met all inclusion criteria (Figure 1) .We included 1 unpublished study, which we had identified on the basis of a published trial protocol92 and included after the authors agreed to share relevant data from the completed trial. The 58 included studies reported 66 distinct trials (Table), 50 of which used randomized designs. The 16 nonrandomized trials comprised 3 quasi-randomized trials40, 67 and 13 controlled before-after trials. Interventions included a median of 3 distinct QI strategies (interquartile range, 2-4). Four trials44, 52-53,61 evaluated interventions using 5 distinct QI strategies, and 1 trial44 evaluated an intervention with components of 6 strategies. Only 5 trials17, 42, 50, 58, 84 evaluated single-strategy interventions. Forty-three (65%) of the included trials were conducted in the United States, and the results of 27 (41%) were published within the last 2 years covered by our search. Postintervention HbA1c values were obtained after a median follow-up period of 13 (interquartile range, 6-18) months. Follow-up HbA1c values were reported for a median of 93% of patients (interquartile range, 82%-100%). However, these rates may overstate the completeness of follow-up, because postintervention patients sometimes differed from preintervention patients (eg, when intervention status was allocated at the clinician level). Also, some studies reported data only for those patients with complete follow-up.
|
|
|
|
Figure 1. Search Strategy and Article Review Process
"Excluded topic" applied to studies focused predominantly on diabetes in pregnancy, type 1 diabetes, or diabetes in children/adolescents, screening for new diagnoses of diabetes, preventing diabetes in high-risk patients, and inpatient management of diabetes. The predominant reason for exclusion under "Other" was omission of key data (eg, numbers of patients or preintervention glycosylated hemoglobin [HbA1c] values94-99), which applied to 8 studies. Full text review was conducted by 2 independent reviewers, with conflicts resolved by consensus among 3 reviewers. EPOC indicates Cochrane Collaboration's Effective Practice and Organisation of Care group.
*More than 1 design could be reported in single article.
|
|
|
|
|
|
|
Table. Key Features of Trials Included in the Meta-analysis
|
|
|
Univariate Analyses of Study Features and Effect Size
Across all included trials, the mean postintervention difference in HbA1c values was –0.42% (95% confidence interval [CI], –0.29% to –0.54%), indicating that the mean postintervention HbA1c value was 0.42% lower in intervention groups than in controls. Here and throughout, percentage values refer to the absolute change in HbA1c value, which is measured in percentage points. Also, from this point on, we refer simply to reductions in HbA1c value, although the outcome is always the postintervention difference between intervention and control groups.
Including the baseline difference between the intervention and control groups in a univariate model exerted minimal effect, but a dichotomous variable for poor baseline glycemic control was a significant predictor. Studies in which the mean baseline HbA1c value was at least 8.0% (45 trials) reported significantly greater reductions in HbA1c values than studies with lower baseline values (0.54% vs 0.20%, respectively; P = .005). Effect sizes did not differ on the basis of country (United States vs other country) or key methodological features (eg, study design, clustered allocation, blinding). However, studies with sample sizes less than the median for all included studies (180 patients) reported substantially greater reductions in HbA1c values compared with trials having sample sizes at least as large as the median (0.61% vs 0.27%; P = .004).
This inverse relationship between effect magnitude and sample size remained significant when we stratified by potential confounders, such as study design and the presence of clustering. We also considered the possibility of confounding by type of strategy—ie, that specific QI strategies might be evaluated in larger studies and might also tend to achieve smaller effects. However, we found no evidence for such a relationship. We also considered the possibility that achieving large improvements in glycemic control might prove more difficult over time and that sample sizes for published evaluations of QI interventions might also have increased over time. We found a very weak, nonsignificant relationship between study year and effect size. Inclusion of study year in the model did not alter the significant inverse relationship between effect magnitude and trial size. We therefore regarded the finding that smaller studies reported larger effect sizes as predominantly reflecting publication bias, with smaller trials likely to be published only if they reported large effects. A conventional funnel plot showed highly significant asymmetry (P = .001), though this finding by itself would not convincingly indicate publication bias given the presence of significant heterogeneity.100
Although effect sizes did not differ significantly between trials with randomized and nonrandomized designs across all trials, differences were apparent within some QI strategies. The results of restricting the analysis to randomized trials are presented with results for the specific categories of QI strategies that showed such differences.
QI Strategies and Glycemic Control
Based on the univariate results, we performed multivariate analyses that controlled for study size and mean baseline HbA1c values of 8.0% or greater. Using this model, 2 QI strategies were associated with mean reductions in HbA1c values of at least 0.50%: team changes (0.67%; 95% CI, 0.43% to 0.91%; n = 26 trials) and case management (0.52%; 95% CI, 0.31% to 0.73%; n = 26 trials) (Figure 2). The threshold of 0.50% is purely descriptive; it was not specified a priori as a cutoff for clinically important effects. In fact, 4 additional strategies—patient education, patient reminders, the use of an electronic patient registry, and clinician education—were associated with effects between 0.40% and 0.50%, and pooled effect sizes were statistically significant for all but 2 of the strategies (clinician reminders and continuous QI) (Figure 2). The observed reductions in HbA1c values were relative to the control groups in the included studies, not to other specific QI strategies.
|
|
|
|
Figure 2. Postintervention Differences in Serum HbA1c Values After Adjustment for Study Bias and Baseline HbA1c Values
Negative estimates favor intervention groups over control groups for the indicated quality improvement (QI) strategy. These estimates were derived from a meta-regression model with adjustment for the effects of study size (at least as many patients as the median among all included studies vs not) and a dichotomous variable that equaled 1 if the mean baseline glycosylated hemoglobin (HbA1c) value was  8.0%. Thus, the point estimate of –0.67% for team changes indicates that, given a large study (ie, having at least as many participants as the median of 180 patients) in which the mean baseline HbA1c values were 8.0% or higher, the intervention group would have a follow-up HbA1c value 0.67% lower than the follow-up value in the control group.
|
|
|
Attributable Effects Associated With Each QI Strategy
The estimates in Figure 2 do not take into account the effects of cointerventions. For each QI strategy, the estimate shown reflects the mean reduction in HbA1c values associated with any intervention that included that strategy as a component. Thus, the mean reduction in HbA1c values associated with, for instance, clinician education, could reflect the effects of any of the other strategies present in interventions that included clinician education. To assess the impact attributable to specific QI strategies, we used the meta-regression model to determine if interventions involving a given strategy achieved significantly greater reductions in HbA1c values than did interventions without that strategy. This analysis did not simultaneously compare all 11 strategies; it compared interventions that used a given QI strategy with those that did not, while also adjusting for the effects of baseline glycemic control and trial sample size. This analysis cannot adjust for possible patterns of association between specific categories of QI strategies.
Two strategies were associated with statistically significant incremental reductions in HbA1c values. Interventions involving team changes reduced HbA1c values by 0.33% more (95% CI, 0.12% to 0.54%; P = .004) than did interventions without team changes, and interventions involving case management reduced HbA1c values by 0.22% more (95% CI, 0.00% to 0.44%; P = .04) than did interventions that did not use case management. None of the other strategies were associated with comparable effects. Patient education, which was present in 38 trials, was associated with an incremental reduction in HbA1c values of 0.15% (P = .20); patient reminders, present in 14 trials, were associated with an incremental reduction of 0.11% (P = .40). The remaining strategies were associated with virtually no incremental reduction in HbA1c values.
Key Components of Interventions
For each of the 26 trials that evaluated case management, we characterized the professional background of the case manager (nurse or pharmacist), the relationship of that person to the clinic (based in the clinic or established working relationship with the clinic vs not), the type of patient contact (predominantly by telephone vs not), and whether or not any medication changes could occur without prior approval from a physician. Only the last feature significantly modified the effectiveness of the intervention. Across the 11 trials18, 45, 47, 60, 63-64,71, 74, 76, 79, 92 in which nurse or pharmacist case managers could make at least some independent medication changes, the interventions reduced HbA1c values by a mean of 0.96% (95% CI, 0.52% to 1.41%), compared with 0.41% (95% CI, 0.20% to 0.62%) for the 15 case management trials that did not include this feature (P = .02 for this comparison) (Figure 3). Across all studies and adjusting for both trial size and baseline HbA1c values, case management that included independent medication changes was associated with a reduction in HbA1c values of 0.80% (95% CI, 0.51% to 1.10%), compared with only 0.32% (95% CI, 0.14% to 0.49%) for all other interventions (P = .002 for this comparison). Thus, interventions in which case managers could make independent medication changes achieved reductions in HbA1c values that were 0.49% greater (95% CI, 0.19% to 0.78%) than those achieved by interventions without this feature.
|
|
|
|
Figure 3. Random-Effects Meta-analysis of Trials Involving Case Management
Across the 26 trials that evaluated interventions that included case management, the pooled reduction in glycosylated hemoglobin (HbA1c) values was 0.59% (95% confidence interval [CI], 0.41% to 0.77%). For the 15 trials in which case managers required physician approval prior to making medication changes, the pooled reduction in HbA1c values was only 0.41% (95% CI, 0.20% to 0.62%), compared with 0.96% (95% CI, 0.52% to 1.41%) for the 11 trials in which nurse or pharmacist case managers could make independent medication changes (P = .003). Adjusting for trial size and baseline HbA1c values altered these results only slightly, with a mean reduction in HbA1c values of 0.86% (95% CI, 0.45% to 1.28%) for case management interventions in which case managers could make independent medication changes, compared with 0.40% (95% CI, 0.02% to 0.78%) for other case management interventions (P = .04). The sizes of data markers indicates the relative weight of each trial in the meta-analysis, which reflects the precision of the estimate from the trial (largely determined by the number of patients in the trial).
|
|
|
For the 26 trials that evaluated team changes, we classified interventions as having 1 or more of the following 4 features: (1) patient care included 1 new team member (ie, patients routinely saw 1 clinician other than the primary physician, such as a diabetes nurse specialist, dietician, or pharmacist); (2) the intervention added more than 1 new team member (eg, a diabetes nurse specialist and a dietician) or was explicitly described as involving a multidisciplinary team; (3) expansion or revision of professional roles (eg, nurse or pharmacist played more active role in patient monitoring or adjusting medication regimens); or (4) the intervention was explicitly identified as "shared care" between specialists and primary care clinicians.
Sixteen interventions involved multidisciplinary teams and conferred an incremental reduction in HbA1c values of 0.37% (95% CI, 0.16% to 0.58%). In other words, adjusting for trial size and mean baseline HbA1c values of 8.0% or greater, interventions that involved multidisciplinary teams achieved reductions in HbA1c values 0.37% greater than interventions without this feature (P<.001). Shared care and expansion or revision of professional roles were associated with comparable effect sizes, though these were not statistically significant, presumably due to small numbers of trials (5 and 4, respectively). However, simply adding a new team member did not confer comparable benefit. On average, these interventions were associated with reductions in HbA1c values that were 0.19% less than those of other interventions, though this difference was not statistically significant (P = .20), again presumably due to lack of power (6 trials).
The definitions of team changes and case management only partially overlapped. Ten trials were classified as involving both case management and team changes, but 16 trials involved case management without team changes and 16 trials involved team changes without case management. We formally tested for confounding by evaluating a meta-regression model that included trial size, mean baseline HbA1c values of 8.0% or greater, and the presence of case management. Team changes remained a significant contributor to the reduction in HbA1c values in this model, conferring an additional reduction of 0.33% (95% CI, 0.13% to 0.53%) beyond that associated with interventions without team changes. Team changes also remained a significant predictor in a model that included the feature of independent medication changes by case managers.
Across all QI strategies, we also explored the effects of the number of QI strategies per intervention. There was no significant trend in effectiveness based on the number of interventions, whether analyzed as an ordinal variable or as a dichotomous one that distinguished multifaceted interventions from single-faceted ones. Interventions that used at least 2 QI strategies (ie, multifaceted interventions) reported reductions in HbA1c values that were only 0.03% greater than those achieved by single-faceted interventions (P = .90), though only 5 trials17, 42, 50, 58, 84 evaluated single QI strategies.
Additional Analyses
Reductions in HbA1c values varied across trials to a greater extent than would be expected by chance alone (ie, the studies showed significant heterogeneity [P<.001]). Such nonrandom variation is expected for an analysis of complex interventions involving multiple different QI strategies and is what motivated the use of a meta-regression approach. Including study size and baseline HbA1c values of 8.0% or greater in the regression model reduced the unexplained variation in effect size across trials by approximately 50%. Adding case management and team changes to the model accounted for all remaining nonrandom variation.
We tested various alternate classifications of the QI categories, which did not substantially alter the results. We also conducted analyses that excluded clustered trials (because of the frequent requirement to impute key data about cluster size and correlation within clusters), which produced little change in the main results, as did restricting the analysis to studies with effective sample sizes of at least 100 patients. Restricting the analysis to randomized trials did alter some of the results, more notably for team changes than for case management. Among randomized trials, the 19 interventions that involved team changes achieved reductions in HbA1c values only 0.17% larger than those achieved by interventions without this strategy (P = .16). For case management, the incremental impact on HbA1c values remained virtually unchanged at 0.23% (95% CI, 0.03% to 0.44%; P = .04). However, the pooled effect was somewhat diminished, with a mean reduction of 0.42% (95% CI, 0.24% to 0.61%) compared with 0.52% (95% CI, 0.31% to 0.73%) for the estimate that included nonrandomized trials. Nonetheless, case management in which case managers could make at least some medication changes without waiting for approval from physicians remained the strategy with the most notable effects, with a mean reduction in HbA1c values of 0.60% (95% CI, 0.28% to 0.92%), compared with only 0.24% (95% CI, 0.08% to 0.41%) for interventions that did not include this strategy.
Adverse Effects of the Interventions
The studies did not consistently or comprehensively assess for the possibility of undesirable effects as a result of the intervention. Taking into account baseline differences in glycemic control, 5 trials reported increases in HbA1c values in the intervention group.43, 50, 76, 86-87 These increases were small (0.2% or less) and nonsignificant. Only 12 trials reported any results related to the occurrence of hypoglycemia. Seven trials18, 44-45,59, 61, 79, 89 reported increases in hypoglycemia for patients in the intervention group, one18 of which was likely statistically significant. Three trials19, 57, 84 reported decreases in hypoglycemia for intervention patients, one19 of which was likely statistically significant. The other 2 trials reported zero events39 or rates that were virtually the same in both groups.42 Variations in reporting formats and the level of supplied detail prevented quantitative analysis of these findings.
COMMENT
We systematically evaluated the effectiveness of 11 categories of interventions designed to improve the outpatient care of patients with type 2 diabetes. After adjustment for potential publication bias and baseline glycemic control, 2 strategies—case management and team changes—were associated with the largest pooled reductions in HbA1c values and were also the only strategies that conferred significant incremental reductions. The interventions involving either of these strategies were associated with significantly larger reductions in HbA1c values than interventions that did not involve them.
We also found that a key ingredient in the success of case management interventions was the ability of case managers to make medication changes without waiting for physician approval. Although this aspect of the analysis was post hoc, the results were striking. Interventions in which case managers could make independent medication changes achieved a mean reduction in HbA1c values of 0.80% (95% CI, 0.51% to 1.10%), compared with only 0.32% (95% CI, 0.14% to 0.49%) for all other interventions (P = .002). Among randomized trials, this effect decreased moderately to 0.60% (95% CI, 0.28% to 0.92%), but so did the results for other strategies. For team changes, we were not able to identify a single effective subtype, but simply adding a new team member likely confers no benefit compared with the use of multidisciplinary teams, shared care between specialists and primary care clinicians, or adding a new team member with an expanded professional role.
Our positive findings for case management are consistent with the results of past reviews of case and disease management,9, 11, 101-102 though previous definitions have been very broad. For instance, one review defined case management as "a set of activities whereby the needs of populations of patients at risk for excessive resource utilization, poor outcomes, or poor coordination of services are identified and addressed through improved planning, coordination, and provision of care."9 Our definition—that at least 1 person other than the patient's physician play an active role in coordinating diagnosis, treatment, or ongoing patient management—more readily translates into an operational definition for classifying interventions and for implementation in practice. Those interested in implementation should review some of the specific studies identified as using case management (Table), especially the 11 trials in which case managers could make medication changes without waiting for physician approval. In 4 of these trials,18, 44, 56, 70 case managers followed treatment protocols that specified only target blood glucose values, but in 5 trials51, 64, 73, 79 the protocols included guidance for medication adjustments. In 4 trials,44, 51, 78 case managers had access to a physician (usually an endocrinologist) who provided general support for the intervention.
Our analysis involves a number of advances over most systematic reviews of QI interventions, including adjustment for baseline status, detection of publication bias and correction for its effects, accounting for the use of clustered designs, and addressing the problems of confounding by cointerventions. Nonetheless, our analysis has important limitations. First, the analysis necessarily involved approximately 20 comparisons—the 11 distinct categories of strategies plus trial features such as sample size, design, country, and study year. Some of our findings would remain statistically significant even if corrected for this many comparisons. However, by its nature, our analysis should be regarded as exploratory and our findings as hypothesis-generating.
A second limitation concerns the intrinsic complexity of the interventions. This complexity, compounded by the paucity of descriptive detail in most studies, may have resulted in misclassification of interventions, which would tend to bias toward the null. Thus, some of the QI strategies for which we found small mean reductions in HbA1c values may have had their true effects dampened. Lack of detail about intervention features represents a frequently encountered problem for systematic reviews of QI strategies.30, 103 As more guidelines for QI research appear,104 published evaluations will hopefully provide more complete descriptions of interventions and do so using established taxonomies.29
Third, despite our attempts to take into account effects of cointerventions and important trial features in our analysis, there remain important confounders and selection effects for which we could not adjust. For instance, patients enrolled in case management programs may have been more likely to show larger reductions in HbA1c values because they tended to have poor glycemic control to begin with and possibly because of other factors such as motivation. There may also have been confounding by important organizational characteristics. For instance, the implementation of an electronic patient registry likely reflects greater institutional investment in QI activities. Finally, the primary studies themselves have important limitations, including modest sample sizes, methodological shortcomings, and variable quality of reporting.
We found that larger studies reported significantly smaller reductions in HbA1c values than did smaller studies. Although routinely addressed in meta-analyses of clinical interventions, few systematic reviews of QI strategies have considered publication bias,11, 26, 31 and no previous review has attempted to adjust for its effects. In an earlier review,22 we found that trial design was significantly associated with effect magnitude for measures of clinician adherence to target processes of care. Nonrandomized trials reported improvements in clinician adherence that exceeded by 14% those observed in randomized trials (P = .001) and also reported significantly larger improvements in glycemic control. The present review found a less striking relationship between trial design and effect magnitude, reflecting publication in the last 2 years of more controlled before-after trials that reported small, nonsignificant changes in HbA1c. values. Nonetheless, within many of the categories of QI strategies, trends toward smaller effects with randomized trials were seen, underscoring the importance of data from randomized controlled trials to evaluate candidate QI interventions.
Our finding that single-strategy interventions achieved comparable effects to multifaceted interventions is not as robust as our other findings, given that the analysis included only 5 single-strategy interventions. It is worth noting, however, that 3 recent systematic reviews of QI strategies—2 reviews across a broad range of conditions24, 105 and 1 in hypertension care106—have found no difference in effect between single-faceted and multifaceted interventions.
CONCLUSIONS
In summary, we found that most QI strategies for diabetes care produced small to modest improvements in glycemic control. Relative to other QI strategies, team changes and case management were associated with larger, more robust reductions in HbA1c values than the other 9 QI strategies evaluated. However, our findings of smaller effects for other QI strategies should be interpreted cautiously, given that some strategies were evaluated in relatively few trials, and heterogeneity or misclassification may have lessened our ability to determine their full effects. It is also worth noting that no interventions involving financial incentives for clinicians met our inclusion criteria. The literature on such incentives is still relatively sparse107 but shows some promise. The current interest in "pay for performance"108 will hopefully generate more evaluations of interventions focused on improving diabetes care.
For those with a purely pragmatic interest in effective means of improving outpatient diabetes care, the most striking finding of our analysis is that case management in which nurse or pharmacist case managers could make independent medication changes was associated with improvements in glycemic control that were substantially larger than those associated with any other strategy. However, the importance of this component of case management was not one of our prespecified hypotheses. Moreover, the selection of case management as an intervention, especially one involving expanded professional roles for case managers, may have been a marker for important patient or organizational features likely to produce larger improvements in glycemic control. The effectiveness of case management should therefore be confirmed in rigorously designed, large, prospective trials. In addition to capturing important clinical outcomes, such trials should include detailed economic evaluation, as the existing literature does not provide sufficient information to determine the cost-effectiveness of implementing specific QI strategies, many of which are resource intensive.
AUTHOR INFORMATION
Corresponding Author: Kaveh G. Shojania, MD, The Ottawa Hospital–Civic Campus, 1053 Carling Ave, Room C403, Box 693, Ottawa, Ontario, Canada K1Y 4E9 (kshojania{at}ohri.ca).
Author Contributions: Dr Shojania had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Shojania, Ranji, McDonald, Grimshaw, Owens.
Acquisition of data: Shojania, Ranji, Sundaram, Owens.
Analysis and interpretation of data: Shojania, Ranji, McDonald, Grimshaw, Sundaram, Rushakoff, Owens.
Drafting of the manuscript: Shojania, Ranji, Owens.
Critical revision of the manuscript for important intellectual content: Ranji, McDonald, Grimshaw, Sundaram, Rushakoff, Owens.
Statistical analysis: Shojania, Ranji, Sundaram, Owens.
Obtained funding: Shojania, McDonald, Owens.
Administrative, technical, or material support: Shojania, McDonald, Sundaram.
Study supervision: Owens.
Determination of cutoff criteria for analysis: Rushakoff.
Financial Disclosures: None reported.
Funding/Support: This article is based in part on work performed by the Stanford-UCSF Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality (Contract 290-02-0017), Rockville, Md. Drs Shojania and Grimshaw hold Government of Canada Research Chairs; Dr Owens receives support from the Department of Veterans Affairs.
Role of the Sponsors: None of the funding organizations had any role in the design and conduct of the study; the collection, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
Acknowledgment: We thank Lauren K. Shaw, Lisa N. Charo, BA, and Jennifer C. Lai, BSc, paid research assistants at the time of the study, for assistance with article reviews and abstractions; Alan Bostrom, PhD, Department of Epidemiology and Statistics, University of California, San Francisco, for statistical expertise; and Robert M. Wachter, MD, Department of Medicine, University of California, San Francisco, and Amy J. Markowitz, JD, and for editorial comments on earlier drafts of the manuscript. Dr Bostrom received hourly compensation for his contributions; Dr Wachter and Ms Markowitz received no compensation. We also thank the authors who responded to questions about their published studies and the additional authors who shared unpublished data.
Author Affiliations: Ottawa Health Research Institute and Department of Medicine (Drs Shojania and Grimshaw) and Institute of Population Health (Dr Grimshaw), University of Ottawa, Ottawa, Ontario; Department of Medicine (Drs Ranji and Rushakoff) and Division of Endocrinology (Dr Rushakoff), University of California, San Francisco; and Center for Primary Care and Outcomes Research, Stanford University (Mss McDonald and Sundaram and Dr Owens) and VA Palo Alto Health Care System (Ms Sundaram and Dr Owens), Palo Alto, Calif.
REFERENCES
|
