 |
|
Effect of Patient Reminder/Recall Interventions on Immunization Rates
A Review
Peter G. Szilagyi, MD, MPH;
Clayton Bordley, MD, MPH;
Julie C. Vann, PhD, MS, RN;
Ann Chelminski, MD;
Ronald M. Kraus, EdM;
Peter A. Margolis, MD, PhD;
Lance E. Rodewald, MD
JAMA. 2000;284:1820-1827.
ABSTRACT
| |
Context Immunization rates for children and adults remain below national goals. While experts recommend that health care professionals remind patients of needed immunizations, few practitioners actually use reminders. Little is known about the effectiveness of reminders in different settings or patient populations.
Objectives To assess the effectiveness of patient reminder systems in improving immunization rates, and to compare the effectiveness of different types of reminders for a variety of patient populations.
Data Sources A search was performed using MEDLINE, EMBASE, PsychINFO, Sociological Abstracts, and CAB Health Abstracts. Relevant articles, as well as published abstracts, conference proceedings, and files of study collaborators, were searched for relevant references.
Study Selection and Data Extraction English-language studies involving patient reminder/recall interventions (using criteria established by the Cochrane Collaboration) were eligible for review if they involved randomized controlled trials, controlled before-after studies, or interrupted time series, and measured immunization rates. Of 109 studies identified, 41 met eligibility criteria. Studies were reviewed independently by 2 reviewers using a standardized checklist. Results of studies are expressed as absolute percentage-point changes in immunization rates and as odds ratios (ORs). Studies with similar characteristics of patients or interventions were pooled (random effects model).
Data Synthesis Patient reminder systems were effective in improving immunization rates in 33 (80%) of the 41 studies, irrespective of baseline immunization rates, patient age, setting, or vaccination type. Increases in immunization rates due to reminders ranged from 5 to 20 percentage points. Reminders were effective for childhood vaccinations (OR, 2.02; 95% confidence interval [CI], 1.49-2.72), childhood influenza vaccinations (OR, 4.25; 95% CI, 2.10-8.60), adult pneumococcus or tetanus vaccinations (OR, 5.14; 95% CI, 1.21-21.78), and adult influenza vaccinations (OR, 2.29; 95% CI, 1.69-3.10). While reminders were most effective in academic settings (OR, 3.33; 95% CI, 1.98-5.58), they were also highly effective in private practice settings (OR, 1.79; 95% CI, 1.45-2.22) and public health clinics (OR, 2.09; 95% CI, 1.42-3.07). All types of reminders were effective (postcards, letters, and telephone or autodialer calls), with telephone reminders being most effective but costliest.
Conclusions Patient reminder systems in primary care settings are effective in improving immunization rates. Primary care physicians should use patient reminders to improve immunization delivery.
INTRODUCTION
Immunization rates for children and adults are rising throughout the United States, but coverage levels have not reached national goals. In 1998, coverage levels for children aged 19 to 35 months were 79% for the combined vaccine series of 4 diphtheria-tetanus-pertussis, 3 polio, 1 measles-containing vaccine, and 3 Haemophilus influenzae type b; 87% for 3 hepatitis B; and 43% for varicella.1 Coverage levels for adults are lower; in 1997, only 65% of adults aged 65 years or older received the influenza vaccine and only 45% had ever received pneumococcal vaccine.2 Furthermore, immunization coverage levels are not evenly distributed, with lower rates occurring among impoverished populations3 and some primary care practices.4 As a result of low immunization rates, vaccine-preventable diseases still occur, as evidenced by the measles epidemic during 1989-1991; the large number of annual cases of varicella, pertussis, and hepatitis B; and the thousands of annual deaths among adults from influenza and pneumococcal infections. Concern also exists that incorporation of new vaccinations will be slow, as evidenced by the slow uptake of varicella vaccine.5
In an era of increasing complexity of immunization schedules, increasing expectations about the performance of primary care, and large demands on primary care physicians, it is important to understand and promote interventions that work in primary care settings. Recent reviews have identified several promising strategies to improve immunization rates.6-10 One strategy recommended by the Task Force on Community Preventive Services9 and the Standards for Immunization Practices11 involves patient reminder/recall systems.
Unfortunately, few primary care physicians actually use reminder/recall systems.12 Because many patients cannot remember the recommended immunization schedule, the burden falls on primary care physicians to ensure that their patients receive immunizations on a timely basis. Recently, the burden on the private sector has increased as more patients have begun to receive immunizations at their primary care physician's office rather than at health department immunization clinics.13
If experts are recommending reminder/recall systems and individual studies are demonstrating their effectiveness, why aren't these systems used more frequently in primary care settings? Several factors may impede their incorporation. First, health care practitioners may not perceive that individual studies apply to their own practices. Pediatricians may not focus on studies involving elderly adults, and internists may not be aware of studies involving children. Some studies have been performed in public health department clinics or academic teaching hospital clinics, and private practitioners may not think that findings from such studies can be applied to their settings. Furthermore, some vaccinations are given only once, while others require multiple booster doses, making it more difficult to extrapolate findings from individual regimens to all vaccinations.
A second barrier is that recommendations about reminder/recall systems have not been very specific.10-11 Patient reminders can be delivered by a variety of methods (eg, telephone, mail) and in different levels of intensity (eg, single or multiple reminders). The most useful recommendations are those that are specific enough to be applicable in real-world settings by large numbers of practitioners. A third barrier is that many primary care practices have lacked the computerized technology to track their patients' immunization status. Cost barriers may have also impeded use of computerized tracking and reminder systems. However, recent advances in billing systems and computerized immunization registries14 are making such technology attainable and affordable for a growing number of primary care practitioners.
We systematically reviewed the literature for studies of patient reminder/recall systems to assess their overall effectiveness and to delineate particular systems and situations that appear to be most effective in improving immunization rates. The study objectives were to (1) assess the overall effectiveness of patient reminder/recall systems in improving immunization rates; (2) compare the effectiveness of reminder/recall systems among populations that varied by baseline immunization rates, age, primary care settings, or vaccination schedules; and (3) compare the effectiveness of different types of reminder/recall interventions (eg, postcard, letter, telephone), and frequency of prompts (eg, single or multiple).
METHODS
We followed the methodological review criteria established by the Cochrane Collaboration.15 Specific formats of the reviews can be found in the Cochrane Reviewers' Handbook,16 and an electronic publication containing detailed information about each study reviewed by this project is forthcoming as part of the Cochrane Database of Systematic Reviews.17
Study Selection
We sought studies involving interventions that reminded patients of immunizations that were due or immunization visits that were upcoming (reminders) or immunizations that were overdue (recall). Reminder/recall systems could be delivered by letter, postcard, telephone, autodialer (a computerized telephone dialer programmed to generate multiple telephone calls during a short time), or in person. Reminder/recall cues could also vary in their specificity (generic or patient-specific) and in their frequency (single or multiple).
The key outcome measure was immunization rates, or the proportion of the target population that was up-to-date on recommended immunizations. We included studies with outcomes for either individual vaccinations or standard combinations of recommended vaccinations (eg, all recommended vaccinations by a specific date or age).
Interventions that involved physician reminders, such as medical chart or computer prompts, were not evaluated unless they were used in combination with patient reminders. Studies with these combined interventions were analyzed separately from studies evaluating only patient reminders.
Three study designs were eligible for review: randomized controlled trials, controlled before-and-after studies, and interrupted time series studies. Studies had to meet initial published design criteria to be eligible for full review.
Search Strategy
A search was performed using the following bibliographic search engines: MEDLINE, EMBASE, PsychINFO, Sociological Abstracts, and CAB Health Abstracts; all databases were searched from their inception dates through 1998. Most studies were identified using MEDLINE. Search terms included the following text words and Medical Subject Headings: remind$, track$, autodial$, postcard$, mail$, recall$, telephone$, registry$, registries, reminder systems, appointments & schedules, appointment$, information systems, computers, immunization, immuniz$, immunization programs, vaccination, vaccin$, innoculat$, prevention health services, diphtheria, tetanus, whooping cough, poliomyelitis, polioviruses, haemophilus, influenza, measles, mumps, rubella, hepatitis b, pneumococcal infections, vaccines, tetanus toxoid, and diphtheria toxoid.
Two authors (P.G.S. and R.M.K.) reviewed the lists of titles and abstracts and used the inclusion criteria to select potentially relevant articles for full review. The reference lists of all relevant articles and reviews were back-searched for additional studies. Publications of abstracts, proceedings from scientific meetings, and files of study collaborators were also searched for references.
Inclusion Criteria
Articles were reviewed if they (1) included a patient reminder/recall system in at least 1 study arm; (2) reported primary research; (3) studied common nationally or internationally recommended childhood or adult vaccines (unusual vaccines or vaccines for travelers were excluded); (4) provided immunization coverage data; and (5) were written in English. Studies examining the effect of patient reminder/recall systems on other preventive services were included only if they reported on immunization rates separately.
Data Abstraction and Review
Each study was read independently by 2 reviewers (P.G.S. and J.C.V.). Reviewers were not blinded to authors (a recent study found no significant bias associated with such nonblinding18). Disagreements between reviewers were resolved by a formal reconciliation process to achieve consensus. Data abstraction was performed using a validated checklist developed by the Cochrane Collaboration Effective Practice and Organization of Care Group.19 For each included study, information was collected on the method of randomization or assembly of control groups, blinding, characteristics of subjects, setting and nature of the interventions, and results. Numerous quality criteria were assessed for each study design. For randomized controlled trials, which were the majority of included studies, assessment criteria included concealment of allocation, proportion of participants followed up, blinded assessment of primary outcome measures, documentation of baseline data, reliability of outcome measures, and protection of contamination between study groups.
The primary outcome measures were the percentage of patients who were immunized at the end of the study and the difference, in absolute percentage points, in immunization rates between groups receiving a reminder/recall vs control groups.
We were interested in both the overall effectiveness of patient reminder/recall and the relative effectiveness for key subgroups defined by patient age (child or adult), practice setting (academic medical center–based clinic, public health department clinic, or private practice), dates of study (before 1980, 1980-1989, or 1990-1998), type of vaccination (universal, such as all routine childhood vaccinations; or targeted, such as influenza for high-risk patients with specific chronic diseases), type of reminder/recall intervention (postcard, letter, telephone, autodialer, or combination), and frequency of intervention (single or multiple).
Analysis
Results are presented for individual studies as absolute changes in immunization rates (eg, >20-percentage-point increase in intervention vs control group rates), rather than relative rates, to allow for comparisons among studies. When possible, odds ratios (ORs) for being up to date or having received vaccinations are shown for intervention vs control patients. Studies were also subgrouped according to the key characteristics described herein. For each subgroup and for all studies combined, summary ORs were obtained using Review Manager, the computer program for analyzing Cochrane Reviews.16
We analyzed the study results as a funnel plot of the effect of reminder/recall against the sample size (which we used as a proxy for study precision since variance in immunization rates was often not available). If publication bias existed, we expected that more precise studies would be more likely to cluster around null results.20 Such clustering was not found, suggesting that positive findings of published studies were not due to publication bias.
Initially pooled results, weighted by the sample size of each study, were calculated using a fixed-effects model. Pooled weighted results were also generated for reminder type, patient age, and major vaccine category. Heterogeneity of the results of individual studies combined for each subgroup comparison was tested using a  2 distribution with a .10 level of significance. Because heterogeneity of the results was present for overall results and within several subgroups, pooled results were ultimately computed using a random-effects model for all comparisons, with studies sorted by key characteristics. The random-effects models had wider 95% confidence intervals (CIs) than the fixed-effect models for all analyses, producing more conservative estimates of the effects of the interventions. In addition, a qualitative analysis (examining the strengths, weaknesses, and unique characteristics of each study) was performed to assess possible factors leading to heterogeneity of results.
RESULTS
Literature Search Results
Ninety-two studies were identified by the literature search. Seventeen additional studies were found by back-searching. Of the total 109 studies, 41 met eligibility criteria and were included in the final review (some studies had more than 1 study arm). The majority of excluded articles lacked a control group, had a study design that did not fit Cochrane criteria (eg, descriptive or ecological studies), were reviews instead of studies, or used an outcome measure other than immunizations (eg, preventive visits or services).
Overall Impact of Reminder/Recall Interventions
We categorized studies into 4 groups according to age (children or adults) and type of immunization: (1) routine childhood immunizations (15 studies21-35); (2) childhood influenza vaccinations that target a high-risk group of children rather than the entire child population (2 studies36-37); (3) adult pneumococcal or tetanus immunizations (7 studies38-44); and (4) adult influenza vaccinations (21 studies38-39,41-42,45-61). Four evidence tables, including summaries of each study, are available from the author by e-mail and will be available on a forthcoming Cochrane review.17
Reminder/recall systems were found to be effective in 33 (80%) of the 41 studies and were generally effective for both children and adults and for both routine immunizations and targeted influenza immunizations. For routine childhood immunizations, 12 of 15 studies found positive effects, with the improvement in immunization rates ranging from 6 to 34 percentage points, and with ORs generally in the range of 1.5 to 2.5 for intervention vs control groups. The 2 studies on childhood influenza immunizations found similar improvements (>20 percentage points) in influenza immunization rates from very low baseline rates of controls. Six of 7 studies of adult pneumococcal or tetanus immunizations reported significant improvements, ranging from 4 to 27 percentage points, with most improving by at least 20 percentage points. Among the 21 studies of adult influenza immunizations (4 of these also studied pneumococcus or tetanus), 5 studies reported no improvement and 16 studies found significant improvements, often greater than 20 percentage points for patients receiving a reminder/recall intervention.
One might expect that improvement in immunization rates would be easier to achieve at lower baseline levels, with diminishing returns at higher levels; however, this was not found. In general, the degree of improvement in immunization rates due to reminder/recall was not associated with baseline immunization levels, which ranged from nearly 0% to 86%, as measured by control groups or assessed at the start of studies. Baseline rates were more than 80% in only a few cases and for influenza vaccine were extremely low.
Effectiveness of Reminder/Recall According to Patient Characteristics
Table 1 shows pooled results of the effectiveness of reminder/recall systems for randomized controlled trials, by patient age and vaccine group, study setting, and decade of study. Reminder/recall was effective for both children and adults, for both universally recommended routine childhood vaccinations and influenza vaccination recommended for high-risk children, and for all types of adult vaccinations. Odds ratios were generally higher than 2.0 for intervention vs control group immunization rates. Reminder/recall was most effective in academic settings, with somewhat lower but still positive results in private practice settings. There was no clear trend by decade of study.
|
|
|
|
Table 1. Effectiveness of Patient Reminder/Recall by Patient Characteristics
|
|
|
Effectiveness of Different Reminder/Recall Systems
Table 2 shows pooled results for randomized controlled trials comparing the effectiveness of different types of reminder/recall systems for routine vaccination of preschool children, child influenza vaccination, adult influenza vaccination, and other adult vaccines. Figure 1 shows ORs for immunization rates for 6 different reminder/recall systems. All types of reminder/recall systems appeared to improve immunization rates, with ORs generally between 1.5 and 2.5 and improvement in immunization rates of 5 to 20 percentage points. Telephone reminders appeared to be more effective than other types of reminders, while letter reminders generally did not appear to have an advantage over postcard reminders. The few studies that evaluated patient reminder/recall combined with physician prompts found results that were similar or slightly better than that of studies using only patient reminder/recall.
|
|
|
|
Table 2. Effectiveness of Different Types of Reminder/Recall Systems for Children and Adults
|
|
|
|
|
|
|
Figure. Effectiveness of Different Types of Patient Reminder/Recall Systems for All Ages Combined
OR indicates odds ratio; CI, confidence interval. Error bars indicate 95% CIs.
|
|
|
Some studies used single patient reminders while others used 2 or more reminders. For the 31 randomized clinical trials that included at least 1 study arm with single patient reminders, the OR for being up to date at the end of the study for intervention vs control groups was 2.18 (95% CI, 1.75-2.71), while for the 9 trials that had multiple reminders, the OR was 2.82 (95% CI, 1.57-5.06).
Costs of Reminder Systems
Fifteen studies reported on costs, including 8 pediatric studies22-23,26-27,31-33,35 and 7 studies of adults.40-41,44, 48, 57, 59, 61 Eight studies estimated the cost-effectiveness of reminder/recall systems.22, 26-27,32, 35, 41, 48, 61 Costs varied widely across studies due to (1) variability in methods of calculating costs and items included in analyses (such as existing staff or computer programming); (2) different types of reminders used, with telephone reminders being more costly than letter or postcard reminders, (3) different levels of intensity of interventions, from single postcard reminders to repeated reminders plus home visits, and (4) different study periods. Single reminders were less costly than multiple reminders but also were less effective. Costs of reminder systems used throughout a year (as in many pediatric studies) were more expensive than short-term reminders typical of influenza vaccination studies. Because of different study methods, it was not possible to combine results of costs. In several studies, the costs per patient were less than $1, particularly in short-term studies involving mailed reminders and excluding computer or study design costs.27, 44, 48, 57, 59 In the few studies that estimated cost-effectiveness, the estimated cost per additional patient vaccinated varied widely, from less than $10 per patient22, 26-27,41 to more than $10 per patient,48 with some reporting even higher costs.32, 35 A short-term pediatric study27 targeting 20-month-old children for receipt of immunizations noted a cost of $9.80 per child appropriately vaccinated by age 24 months using an autodialer reminder, $10.50 per child using a letter reminder, and $7 per child using a combination of approaches. A more comprehensive pediatric intervention involving patient reminders and outreach throughout the year noted a cost of $63 per child per year, with a cost-effectiveness of $316 per year per additional fully vaccinated child.35 Several studies noted that patient reminder systems had the added benefit of increasing preventive visits and receipt of preventive services in addition to immunizations,32-33,35 making cost-effectiveness difficult to assess but increasing the benefits of the immunization intervention.
Analysis of Studies That Found No Improvements
Since 20% of studies did not find improvements in immunization rates due to patient reminder/recall systems, we performed a qualitative analysis of study characteristics for the 8 studies that noted no improvements.25, 31, 33, 39, 54, 56, 59-60 Three of these studies involved children. Seven studies involved mailed reminders and 1 evaluated an autodialer reminder.31 Authors or reviewers noted the following concerns that might have contributed to the failure of these studies to find significant effects: 1 study clearly focused more on a 15-minute postpartum educational discussion than the reminder25; 2 studies used reminders targeting preventive visits rather than immunizations33, 39; 2 studies had small sample sizes31, 39; 2 studies noted significant improvements in 1 subgroup but not another54, 60; in 1 study, the authors noted a possible ceiling effect with high baseline influenza immunization rates (55%-65%) among controls59; and in 1 study, reviewers noted extremely low immunization rates (<2% for adult tetanus),39 suggesting that other factors were somehow impeding immunization delivery. This qualitative analysis did not find clear trends among these studies in terms of practice setting, patient population, or type of reminder/recall used.
COMMENT
The findings from this systematic review of the literature support the general recommendation8-11 that all primary care practitioners should consider patient reminder/recall systems to improve immunization coverage levels of their practices. We found that reminder/recall was effective for both children and adults; in all types of medical settings, including private practices, academic medical centers, and public health department clinics; and for universally recommended vaccinations such as routine childhood vaccinations as well as targeted vaccinations such as influenza vaccine. In addition, all types of patient reminder/recall systems were found to be effective, with increases in immunization rates tending to be 5 to 20 percentage points. Telephone reminders were most effective, while there were no major differences in effectiveness among different types of mailed reminders. More intensive reminder/recall systems, such as those using multiple reminders, appeared to be more effective than single reminders. In studies that evaluated costs, patient reminder systems required a nontrivial expense but led to spillover benefits by increasing preventive visits or receipt of other preventive services.
This study has several limitations. First, we used the Cochrane criteria for selecting studies based on study design and methodological criteria, and some studies were excluded because they did not meet these rigorous criteria. While this strategy improved our ability to estimate the true impact of patient reminder systems, it is possible that some excluded studies may have had different findings and that the impact of reminder systems may be different when rigorous study conditions are not used.
Second, the scope of the review was limited to studies published in English. At least 1 study has found that randomized controlled trials published in English were more likely to have positive findings than studies published in German journals.62 However, such language bias was not noted in other studies.63 In addition, 9 studies included in the current review were performed outside the United States; all 9 studies found positive effects of reminder/recall systems.
A third potential limitation involves publication bias: the majority of studies were located from MEDLINE or references from other studies. Because publication bias typically results in failure to publish studies with negative or null findings,64-66 it is possible that our findings of positive outcomes in 33 of 41 reviewed studies is partly affected by publication bias and that the impact of reminder/recall is lower than noted in this review. We attempted to minimize publication bias by searching the files of the investigators and immunization experts, searching references of published reviews for abstracts, and reviewing abstracts or proceedings of major scientific meetings. In addition, the funnel plot analysis discussed herein did not find that more precise studies clustered around null results, thereby increasing the plausibility of the positive findings.
A fourth set of limitations involves aspects of the systematic review process. We grouped studies according to key characteristics of either the patient population or the intervention. We defined these groups a priori, and they represent standard groupings used in other studies. However, it is possible that where differences were noted by group, factors other than the intervention might have accounted for these differences. Limitations of the standard Cochrane review criteria are published elsewhere.67
Our method of pooling data has limitations, particularly in light of the heterogeneity of some of the data that is often present in meta-analyses.68-69 Because these reminder/recall studies were performed for a variety of populations, using different interventions, in multiple settings, and across 3 decades, it is not surprising that there is interstudy heterogeneity in the results. Because of this heterogeneity, we performed a qualitative analysis of study characteristics that might explain differences in findings among the 8 studies that had negative findings, and although it was easy to find explanations for the negative findings in each study, we did not note consistent trends. We used random-effects analyses, which had consistently more conservative results (wider 95% CIs) than fixed-effects models. In 1 subgroup (adult influenza vaccinations), a single study by Baker et al57 had more than 24,000 subjects and small but significant effects of reminder/recall, while most of the other studies in that group had clinically larger positive effects of reminder/recall. The large sample size and small effect of the 1 study resulted in heterogeneity within this subgroup but also resulted in a conservative effect on the pooled results by reducing the overall impact of reminder/recall.
Because patient reminder/recall systems appear to be effective in all settings that were evaluated, we recommend that all primary care physicians seriously consider incorporating reminder/recall into their practices. Physicians should review the different types of reminder/recall systems and tailor systems to their own needs. While telephone reminders are most effective, they are also more costly and have not been studied extensively in children except for the use of autodialers, which were found to have smaller but positive effects. Practical issues relevant to choices of the reminder/recall system include characteristics of current computer systems, staffing, accuracy of patient telephone numbers or addresses, availability of computer programmers, and estimated patient responsiveness to different types of reminders. Practitioners today can tailor their billing systems to function as reminder/recall systems for simple procedures, such as selecting all patients aged 65 years or older for reminders about influenza or pneumococcal vaccination. Many billing systems have recently incorporated separate modules that can track immunization status.
A critical issue involves the complexity of rules required for a reminder/recall system. The simplest scenario involves elderly adults, because no special immunization algorithm is needed and eligible patients can be selected by birth dates. A slightly more complex scenario involves "flagging" patients with chronic problems, such as asthma, that would require influenza vaccination. More sophisticated algorithms are required to track prior immunization status, particularly for the complicated pediatric immunization schedule. A promising route involves practitioners linking with computerized immunization registries that are being developed throughout the United States. These registries already contain the necessary algorithms to assess up-to-date status of children and could be modified to deliver patient reminders. Finally, databases of managed care organizations can be modified to become reminder/recall systems. For practitioners, the usefulness of such databases depends on the proportion of a practice's patients covered by the managed care plan and the accuracy of the database information.
Overall, the technology exists to incorporate patient reminder/recall into routine primary care practice. There are additional benefits to the patient and practice beyond improving immunization rates. Studies have shown that patients who are behind in immunizations are also behind in other measures of preventive care70-71 and that reminder/recall systems targeting immunizations can also have spillover effects to improve other aspects of preventive care35 if they are used within primary care practices. Second, patients generally appreciate being reminded by their physician, and such reminder/recall systems may actually improve the patient-physician relationship. Third, in fee-for-service settings, patient reminder/recall systems can increase revenues by increasing visits.
Since patient reminder/recall systems for immunizations have been shown to be effective in a variety of settings, we recommend that future research be focused on implementation issues: how to implement reminder/recall in an efficient manner across large numbers of practices, means to effectively use computerized registries for patient reminder/recall, and demonstration on a large-scale population level of whether these interventions improve immunization rates. These questions will be particularly important for certain new vaccines, such as the conjugate pneumococcal vaccine for children or seasonal vaccines (like influenza vaccine), for which "catch-up" strategies or timing issues become important.
The use of patient reminder/recall systems provides the primary care practitioner with real-life experience at practicing population-based care by improving the care for the entire population served by the practice. Although medicine is traditionally taught and practiced one patient at a time, and preventive services such as immunizations are delivered to individual patients, the measures of success (such as immunization rates) are population-based. Such population-based primary care, while not easy to practice in a busy setting, has the potential to improve the quality of care and performance of primary care practitioners.
AUTHOR INFORMATION
Funding/Support: This study was supported by grant U66/CCU312166 from the Centers for Disease Control and Prevention and the Association of Teachers of Preventive Medicine.
Previous Presentations: Presented in part at the 1998 Annual Meeting of the Pediatric Academic Societies, May 4, 1998, New Orleans, La, and at the 1998 National Immunization Conference, July 22, 1998, Atlanta, Ga.
Acknowledgment: We appreciate the assistance of Edward Marcuse, MD, Diane Kittredge, MD, Paul Darden, MD, Diane Langkamp, MD, Carol Lancaster, Sharon Humiston, MD, MPH, and Joseph Pascoe, MD.
Corresponding Author: Peter G. Szilagyi, MD, MPH, University of Rochester School of Medicine and Dentistry, Box 632, Strong Memorial Hospital, 601 Elmwood Ave, Rochester, NY 14642 (e-mail: peter_szilagyi{at}urmc.rochester.edu).
Reprints not available from the author.
Author Affiliations: Department of Pediatrics, University of Rochester School of Medicine and Dentistry, Rochester, NY (Dr Szilagyi and Mr Kraus); the Children's Primary Care Research Group, Department of Pediatrics (Drs Bordley, Vann, Chelminski, and Margolis), and the Public Health Leadership Program, School of Public Health (Dr Vann), University of North Carolina, Chapel Hill; and the National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Ga (Dr Rodewald).
REFERENCES
| |
1. Centers for Disease Control and Prevention. Vaccination coverage levels among children born from February 1995–May 1997—United States, January 1998–December 1998. Available at: http://www.cdc.gov/nip/coverage/nis1998_1.htm. Accessed July 2000.
2. Centers for Disease Control and Prevention. Influenza and pneumococcal vaccination levels among adults aged greater than or equal to 65 years—United States, 1997. MMWR Morb Mortal Wkly Rep. 1998;47:797-802.
PUBMED
3. Centers for Disease Control and Prevention. Vaccination coverage by race/ethnicity and poverty level among children aged 19-35 months—United States. MMWR Morb Mortal Wkly Rep. 1997;47:956-959.
4. Massoudi MS, Walsh J, Stokley S, et al. Assessing immunization performance of private practitioners in Maine: impact of the assessment, feedback, incentives, and exchange strategy. Pediatrics. 1999;103:1218-1223.
FREE FULL TEXT
5. Centers for Disease Control and Prevention. Notice to readers: national vaccination coverage levels among children aged 19-35 months—United States, 1998. MMWR Morb Mortal Wkly Rep. 1999;48:829-830.
PUBMED
6. Gyorkos TW, Tannenbaum TN, Abrahamowicz M, et al. Evaluation of the effectiveness of immunization delivery methods. Can J Public Health. 1994;85(suppl 1):S14-S30.
7. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3:399-409.
FREE FULL TEXT
8. Udovic S, Lieu TA. Evidence on office-based interventions to improve childhood immunization delivery. Pediatr Ann. 1998;27:355-361.
ISI
| PUBMED
9. Task Force on Community Preventive Services. Vaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults. MMWR Morb Mortal Wkly Rep. 1999;48:1-15.
PUBMED
10. Shefer AM, Briss PA, Rodewald L, et al. Improving immunization coverage rates: an evidence-based review of the literature. Epidemiol Rev. 1999;21:96-142.
FREE FULL TEXT
11. Ad Hoc Working Group for the Development of the Standards for Pediatric Immunization Practices. Standards for pediatric immunization practices. JAMA. 1993;269:1817-1822.
FULL TEXT
|
ISI
| PUBMED
12. Szilagyi PG, Rodewald LE, Humiston SG, et al. Immunization practices of pediatricians and family physicians in the United States. Pediatrics. 1994;94:517-523.
FREE FULL TEXT
13. Szilagyi PG, Shone LP, Humiston SG, Kolasa MS, Rodewald LE. Staying in the medical home: decline in physician-reported referrals to health department clinics for immunizations. Am J Prev Med. 2000;18:318-324.
FULL TEXT
|
ISI
| PUBMED
14. Linkins RW, Feikema SM. Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann. 1998;27:349-354.
ISI
| PUBMED
15. Bero L, Rennie D. The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA. 1995;274:1935-1938.
FULL TEXT
|
ISI
| PUBMED
16. Clark M, ed, Oxman AD, ed. Cochrane Reviewers' Handbook 4.0. In: Review Manager [computer program]. Version 4.0. Oxford, England: The Cochrane Collaboration; 1999. Available at: http://www.cochrane.dk/cochrane/handbook/handbook.htm.
17. Szilagyi PG, Bordley C, Margolis P, Kraus R, Chelminski A, Vann J. Interventions to improve immunization rates (protocol for a Cochrane Review). In: The Cochrane Library [database on CD-ROM]. Oxford, England: Update Software; 2000. Issue 3.
18. van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998;280:234-237.
FREE FULL TEXT
19. The Cochrane Collaboration. Reviews for the Cochrane Collaboration on Effective Professional Practice (CCEPP). In: Bero L, Grilli R, Grimshaw J, Oxman A, eds. The Cochrane Collaboration on Effective Professional Practice Module of the Cochrane Database of Systematic Reviews [database on CD-ROM ]. Oxford, England: Update Software; 1999. Issue 3.
20. Poole C, Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol. 1999;150:469-475.
FREE FULL TEXT
21. Alto WA, Fury D, Condo A, Doran M, Aduddell M. Improving the immunization coverage of children less than 7 years old in a family practice residency. J Am Board Fam Pract. 1994;7:472-477.
22. Young SA, Halpin TJ, Johnson DA, Irvin JJ, Marks JS. Effectiveness of a mailed reminder on the immunization levels of infants at high risk of failure to complete immunizations. Am J Public Health. 1980;70:422-424.
FREE FULL TEXT
23. Linkins RW, Dini EF, Watson G, Patriarca PA. A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adolesc Med. 1994;148:908-914.
ABSTRACT
24. Tollestrup K, Hubbard BB. Evaluation of a follow-up system in a county health department's immunization clinic. Am J Prev Med. 1991;7:24-28.
ISI
| PUBMED
25. Oeffinger KC, Roaten SP, Hitchcock MA, Oeffinger PK. The effect of patient education on pediatric immunization rates. J Fam Pract. 1992;35:288-293.
ISI
| PUBMED
26. Lieu TA, Black SB, Ray P, et al. Computer-generated recall letters for underimmunized children: how cost-effective? Pediatr Infect Dis J. 1997;16:28-33.
FULL TEXT
|
ISI
| PUBMED
27. Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics. 1998;101:E3.
28. Ferson MJ, Fitzsimmons G, Christie D, Woollett H. School health nurse interventions to increase immunisation uptake in school entrants. Public Health. 1995;109:25-29.
FULL TEXT
|
ISI
| PUBMED
29. Alemi F, Alemagno S, Goldhagen J, et al. Computer reminders improve on-time immunization rates. Med Care. 1996;34(suppl):OS45-OS51.
30. LeBaron CW, Starnes D, Dini EF, Chambliss JW, Chaney M. The impact of interventions by a community-based organization on inner-city vaccination coverage, Fulton County, Georgia, 1992-1993. Arch Pediatr Adolesc Med. 1998;152:327-332.
FREE FULL TEXT
|
