 |
|
Statewide System of Electronic Notifiable Disease Reporting From Clinical Laboratories
Comparing Automated Reporting With Conventional Methods
Paul Effler, MD, MPH;
Myra Ching-Lee, MPH;
April Bogard, MPH;
Man-Cheng Ieong, MS, MPH;
Trudi Nekomoto, MS;
Daniel Jernigan, MD, MPH
JAMA. 1999;282:1845-1850.
ABSTRACT
| |
Context Notifiable disease surveillance is essential to rapidly identify and respond to outbreaks so that further illness can be prevented. Automating reports from clinical laboratories has been proposed to reduce underreporting and delays.
Objective To compare the timeliness and completeness of a prototypal electronic reporting system with that of conventional laboratory reporting.
Design Laboratory-based reports for 5 conditions received at a state health department between July 1 and December 31, 1998, were reviewed. Completeness of coverage for each reporting system was estimated using capture-recapture methods.
Setting Three statewide private clinical laboratories in Hawaii.
Main Outcome Measures The number and date of reports received, by reporting system, laboratory, and pathogen; completeness of data fields.
Results A total of 357 unique reports of illness were identified; 201 (56%) were received solely through the automated electronic system, 32 (9%) through the conventional system only, and 124 (35%) through both. Thus, electronic reporting resulted in a 2.3-fold (95% confidence interval [CI], 2.0-2.6) increase in reports. Electronic reports arrived an average of 3.8 (95% CI, 2.6-5.0) days earlier than conventional reports. Of 21 data fields common to paper and electronic formats, electronic reports were significantly more likely to be complete for 12 and for 1 field with the conventional system. The estimated completeness of coverage for electronic reporting was 80% (95% CI, 77%-82%) compared with 38% (95% CI, 37%-39%) for the conventional system.
Conclusions In this evaluation, electronic reporting more than doubled the total number of laboratory-based reports received. On average, the electronic reports were more timely and more complete, suggesting that electronic reporting may ultimately facilitate more rapid and comprehensive institution of disease control measures.
INTRODUCTION
Communicable disease surveillance is essential for protecting public health.1-2 State health departments rely on disease reports from clinicians and laboratories to rapidly identify and respond to outbreaks so that additional illness can be prevented.3-6 Recently, the Institute of Medicine cited "a glaring need" to strengthen disease surveillance to prepare for the threat of bioterrorism and address emerging pathogens.7 Previous evaluations of notifiable disease surveillance in Hawaii and elsewhere have found that reports are often submitted late and that communicable illnesses are substantially underreported.5-6 Most disease reports received by state health departments originate from clinical laboratories.8-11 Because the conventional methods of reporting via mail, facsimile, or telephone require active participation of laboratory staff, automated reporting from clinical laboratories has been proposed as a means to improve the quality and timeliness of disease notification.12-13 This report describes the first statewide system of electronic laboratory reporting (ELR) that uses new intercomputer communication technologies.
The Hawaii Department of Health (HDOH) has developed a prototypal laboratory-based electronic communicable disease reporting system (ECDRS) as part of a national effort to promote ELR by the Centers for Disease Control and Prevention.14-15 The ECDRS currently incorporates the 3 largest commercial clinical laboratories in Hawaii, which together serve all geographic areas and generate 60% to 70% of all laboratory-based notifiable disease reports received annually at HDOH.9 Operational since July 1998, the ECDRS remains a work-in-progress. Until it is finalized, HDOH has required the laboratories to submit notifiable disease reports simultaneously through both electronic and conventional reporting mechanisms. We present the results of an evaluation of disease reports received from laboratories participating in the ECDRS, in which we compared the timeliness and completeness of electronic and conventional reporting systems.
METHODS
Disease Reporting Systems Evaluated
The HDOH requires laboratories to report positive test results for 48 communicable pathogens, called notifiable conditions. Traditionally, this reporting system has relied on laboratory staff, usually microbiologists, technicians, or infection control personnel, to recognize notifiable test results and to transcribe specific information regarding the laboratory findings, the patient, and the health care provider onto a standard form. Once completed, the forms are either mailed or sent via facsimile to HDOH. We defined this process as the conventional reporting system.
By contrast, the ECDRS uses automated data extraction and electronic communications to report notifiable conditions to HDOH. At a predetermined time each day, a computer dedicated for reporting purposes in each of the participating laboratories automatically connects to the laboratory information system in routine use at that facility. This connection launches a data extraction program stored within the laboratory information system that scans the admission-discharge-transfer file for test results posted the previous day. The extraction program then searches for records containing codes that identify particular laboratory procedures, and the accompanying "results" section of the record is compared with a data dictionary specific for that procedure. If the test results fulfill the criteria in the data dictionary, the record is flagged and written to a transfer file. The transfer file record contains the test results, details about the specimen, and the patient and health care provider identifiers available within the admission-discharge-transfer file. Following the extraction process, the transfer file is sent to the ECDRS computer, where it is encrypted using commercial software. At a set time each day, this computer connects via modem to a host computer server on standby at HDOH. The laboratory's identity is authenticated by a password before the file is transmitted to a restricted location on the host computer server. Authorized staff subsequently decrypt the file and review each record for content and accuracy.
Evaluation of the ECDRS
To assess the consistency and accuracy of the data extraction and transmission process, all disease reports received through the ECDRS between July 1 and December 31, 1998, were reviewed by a single staff person to determine (1) the total number of records transmitted daily from each laboratory, (2) whether the results reported met the criteria for a notifiable condition, and (3) whether the report was unique or a duplicate. Conditions were considered not notifiable if the agent reported was not on the list of pathogens required to be reported in Hawaii, if the test results for a reportable pathogen were negative, or if the pathogen was required to be reported only under certain circumstances and the criteria for this were not met (eg, isolation from a normally sterile site). A report was considered a duplicate if it contained the same patient name, collection date, specimen number, and test result. Nonduplicate reports were termed unique.
Comparison of Reporting Systems
In a separate analysis, electronic reports received via the ECDRS were compared with those received through the conventional reporting system during the same 6-month evaluation period for 5 notifiable conditions: Salmonella species, Shigella species, Giardia species, vancomycin-resistant Enterococcus, and Streptococcus pneumoniae isolated from a normally sterile site. These pathogens were selected for this evaluation because monitoring them is important for public health, but their occurrence is not so rare or alarming as to generate special attention from laboratory personnel. Moreover, the identification of these agents represents the end point of a variety of different laboratory procedures (eg, culture, microscopy) and requires varying degrees of data synthesis within the laboratory before a report is made (linkage to antimicrobial resistance and/or anatomic site information). The main outcome measures assessed were (1) the number and proportion of reports received by reporting system, laboratory, and pathogen; (2) the date reports were received at HDOH; and (3) the frequency with which a selected set of requested data elements was provided.
Statistical Analysis
Since neither reporting system accounted for 100% of all the reports received, standard Chandra Sekar-Deming capture-recapture methods were used to estimate the total number of reports available for reporting at the laboratory level.16 Completeness of coverage was defined as the total number of reports actually received by each reporting mechanism, divided by the Chandra Sekar-Deming estimate of the number of reports available. The Fisher exact test and the Mantel-Haenszel test with Yates correction were used to determine the significance of the ratio of 2 proportions; confidence intervals (CIs) for mean values were calculated using Epi Info version 6.04c (Centers for Disease Control and Prevention, Atlanta, Ga).
RESULTS
Consistency and Accuracy of the Data Extraction and Transmission Process
Electronic reports were successfully transmitted from the 3 participating laboratories for an average of 133 (72%) of 184 days during the 6-month evaluation period (range, 118-154 days). A total of 15,830 distinct records were received through the ECDRS; of these, 3183 (20%) were duplicate reports and 3520 (22%) were test results not notifiable in Hawaii. The remaining 9127 records (58%) represented unique reports of test results for all notifiable conditions combined.
Comparison of Electronic and Conventional Reports
For the 5 notifiable conditions in the comparison study, a total of 357 unique reports of illness were identified, of which 325 (91%) were reported through the ECDRS system and 156 (44%) were reported via conventional means. As illustrated in Figure 1, 201 (56%) of the notifications were received solely through the ECDRS, 124 (35%) were received through both reporting systems, and 32 (9%) were identified only through the conventional mechanism. Dividing the total number of unique reports by the number received via the conventional system reveals that electronic reporting resulted in a 2.3-fold (95% CI, 2.0-2.6) increase in reports over those obtained through the conventional reporting system alone.
|
|
|
|
Figure. Laboratory-Based Reports Received for 5 Notifiable Conditions by Reporting System, July 1–December 31, 1998
Estimated number of reports available at the laboratory level (n = 408) was derived from the number of reports received through each reporting system and the number reported through both systems using Chandra Sekar-Deming capture-recapture methods. There were a total of 357 unique reports of illness.
|
|
|
Completeness-of-coverage estimates for both reporting systems are presented by notifiable condition and laboratory in Table 1. In these analyses, electronic reporting accounted for a significantly greater proportion of reports for each of the 5 conditions and for reports originating in 2 of the laboratories (A and B); for the third laboratory (C), the number of reports received electronically equaled the number received via the conventional system. Overall, the electronic reporting system captured 80% of the estimated number of reports available at the laboratory level vs 38% for conventional reporting methods.
|
|
|
|
Table 1. Completeness of Coverage* for Electronic and Conventional Laboratory-Based Disease Surveillance Systems by Notifiable Condition and Laboratory
|
|
|
For the 124 reports received through both reporting mechanisms, electronic reports were received an average of 3.8 (95% CI, 2.6-5.0) days earlier than were the corresponding reports sent by mail or facsimile.
In general, electronic reports contained more complete information on the variables common to both conventional and electronic reporting formats. For 2 fields, those indicating patient last name and test result, reports from both systems had complete information. For 17 other fields, reports received electronically were more likely to be complete than were those received via conventional methods; this difference was statistically significant for 12 of the fields (Table 2). Conventional reports had more complete information for the remaining 2 fields, those indicating physician ZIP code and specimen type, although the difference was significant for the latter only.
|
|
|
|
Table 2. Data Field Completion Rates for Reports Received by Electronic (n = 325) and Conventional (n = 156) Reporting Systems
|
|
|
COMMENT
This is the first report regarding a statewide system of automated electronic notifiable disease reporting from private clinical laboratories. For the notifiable conditions included in this evaluation, electronic reporting more than doubled the number of laboratory-based reports received. Such a substantial improvement in notifiable disease reporting could have major implications for public health, resulting in enhanced institution of disease control measures and more extensive disease outbreak investigations.
On average, electronic reports arrived several days prior to conventional reports. They were also more likely to contain key information, such as the telephone numbers for the patient and physician. These observations are important because early reports that have appropriate contact information should enable local health authorities to respond in a more timely manner. While our study was limited to 5 pathogens and, therefore, may not be representative of all laboratory-based notifiable conditions, the consistency of our finding that electronic reports outnumbered conventional reports for each of these agents suggests it may be generalizable to other laboratory results.
Along with the potential benefits of electronic reporting, this report documents challenges that remain to building dependable and efficient automated systems. During the evaluation, electronic transmissions were not received for almost 30% of the days reviewed. The consistency of reporting varied by laboratory and, in general, improved during the 6-month period (data not shown). Lapses in data transmission resulted from a variety of causes, including ongoing adjustments to the data extraction program and failure of the host computer server to reset after suboptimal modem connections. We did not adjust the percentage of all reports received electronically to account for days in which the electronic system did not transmit records, because we wanted to assess the de facto effect of the ECDRS system on reports received at HDOH during the study period. Our estimate that 80% of all notifiable laboratory reports could be captured and transmitted electronically is therefore conservative.
Reporting of extraneous records was problematic, as more than 40% of all reports received through the ECDRS were either duplicate records or results that were not notifiable in Hawaii. The majority of the nonnotifiable reports received were culture results for which reporting is not required (ie, Streptococcus pneumoniae isolated from a site that is not normally sterile, such as the throat). Improving the ability of the data extraction programs to exclude nonnotifiable results has been complicated by the diversity of coding schemes in use among the laboratories. Although each of the participating laboratories uses a major commercial laboratory information management system, these systems are highly configurable, and the test coding schemes in use at each facility have been individualized. Moreover, within each laboratory, the text placed in the test results field to identify a particular laboratory finding may vary. Therefore, the data extraction program and the data dictionary used in the ECDRS must be tailor-made for each facility. While we anticipate that the problem of nonnotifiable results can ultimately be resolved through refinements to the data extraction programs, our experience working in 3 different laboratory environments suggests that implementation of ELR systems would be greatly facilitated by widespread adoption of message standards and standardized coding schemes (eg, Health Level Seven, Laboratory Observation Identifier Names and Codes, and Systemized Nomenclature for Medicine).15, 17-20 Without adoption of such standards, larger public health jurisdictions may find the "custom-built" approach used in Hawaii impractical.
Automating the elimination of duplicate reports will require imaginative programming solutions. Many notifiable disease reports are based on culture isolation and preliminary test results are reported at regular intervals during several days until the final test result is posted. Attempting to solve this problem by expunging all earlier test results for a given specimen is not optimal, because a single serum or stool specimen may be used for several different tests and information could be lost. In addition, while transmitting only final results would decrease the number of duplicate interim reports, earlier, albeit preliminary notification might allow for more timely public health intervention when indicated. For the immediate future, it appears that the deduplication process will require that records be individually reviewed by an experienced staff member.
Two potential concerns about the validity of this evaluation should be addressed. The first is that the relative benefits of electronic reporting cited here might actually be secondary to deterioration of the conventional reporting system. This could have occurred if the participating laboratories had relaxed their efforts to report through conventional means once implementation of the electronic reporting system had begun. To assess this retrospectively, we compared the number of laboratory reports received through the conventional system for the 5 notifiable conditions during the 6-month periods both before and after initiation of the ECDRS. During the period prior to initiation of electronic reporting, HDOH received 150 conventional reports for these 5 conditions from all laboratories in the state, of which 95 (63%) originated from the 3 laboratories participating in the ECDRS. In the period following initiation of electronic reporting, HDOH received 233 conventional reports for these conditions from all laboratories, of which 156 (67%) originated from the 3 participating laboratories. In other words, the proportion of all conventional reports that were submitted by the laboratories participating in the ECDRS remained stable, indicating that conventional reporting from these laboratories had not deteriorated once electronic reporting was initiated. This is not surprising, given that senior management and information management staff at the laboratories were aware that the legal obligation to report via conventional means would remain in effect while the ECDRS was being implemented.
An additional argument against attributing our findings to a deterioration of conventional reporting is the recognition that the conventional and electronic reporting systems operate quite independently. Historically, conventional disease reporting has been the responsibility of staff working at the bench level; the capacity to report electronically, on the other hand, has been developed by computer programmers who are separated organizationally, and often physically, from the areas of the laboratory where microbiology-related activities take place. In the laboratories participating in the ECDRS, the staff responsible for reporting diseases through the electronic and conventional systems perform their duties in relative seclusion from each other.
Another potential concern is that the electronic reporting system might have accounted for a greater number of illness reports because the data extraction and manipulation processes involved inadvertently created specious laboratory records that were then transmitted to HDOH. Although the diagnoses reported were not validated with repeat microbiologic testing, persons identified by laboratory reports as having shigellosis, salmonellosis, or giardiasis were interviewed by HDOH staff per routine procedure. We did not identify a single instance in which the interview failed to corroborate the validity of the electronic report (ie, the individual denied being evaluated for a clinically compatible illness, indicated that he or she did not provide an accordant specimen, or stated a different test result had been given to him or her by his or her health care provider). It seems highly improbable that mistakes with the automated data extraction could have resulted in erroneous yet completely plausible organism names being placed in the test results field; it is much more likely that the electronically reported results accurately reflect laboratory findings.
Other agencies considering developing ELR systems should note that the resources necessary to implement the ECDRS have been significant. Although we hope to have all laboratories in Hawaii included in the ECDRS eventually, we chose to begin with the highest-volume facilities to maximize the potential benefit for the level of effort required. In our experience, the costs for the computer hardware and commercial software involved are minimal, whereas the personnel resources required to develop this prototypal system have been substantial. At each of the participating laboratories, information management staff have had to be integrally involved in writing and refining the data extraction programs as well as solving other technical problems when they occur. This level of commitment has been undertaken without material compensation from HDOH, largely because these laboratories realize that ELR has the potential to improve the quality of disease reporting and, in the long-term, may be a cost-saving alternative to labor-intensive conventional methods. At HDOH, coordinating, monitoring, and troubleshooting the ECDRS project has required the nearly full-time attention of 1 professional staff member who has experience with computers and disease surveillance. This employee is assisted by data systems analysts under contract who make adjustments to the communications system and recommendations to the programmers at the laboratory on a regular basis. When considering costs, however, it is important to recognize that the resources required to develop an archetypal reporting system may be different than those needed to implement ELR once the requisite technologies have become better established. Incorporating the capacity to create ELR messages into commercial laboratory information system software may eventually permit wider, more economical adoption of electronic disease reporting.
At present, laboratories are required to report an average of 45 diseases or conditions under national surveillance to state public health authorities.21 If ultimately implemented across the nation with results similar to those reported here, ELR systems will likely have a positive impact on national morbidity figures and lead to a better understanding of communicable disease epidemiology. Given the increasing importance of laboratories in the fight against infectious diseases,22-23 applying neoteric communication technologies to laboratory-based reporting should enhance our efforts to identify emerging pathogens more comprehensively and respond to potential acts of bioterrorism more rapidly.1, 7, 12 Finally, the value of ELR will be determined by the extent to which it results in public health action at the local level. Future work should assess whether ELR systems improve the ability of health care providers and public health departments to prevent illness through more timely and comprehensive implementation of disease control measures.
AUTHOR INFORMATION
Funding/Support: This work was supported through Cooperative Agreement No. U50/CCU912395-01 from the Centers for Disease Control and Prevention.
Disclaimer: The authors have no commercial, proprietary, or financial interest in the products or companies described in this article. The views expressed herein are those of the authors and do not necessarily reflect those of the State of Hawaii Department of Health or the Centers for Disease Control and Prevention, US Public Health Service, Atlanta, Ga.
Acknowledgment: This work would not have been possible without the dedication of the information management departments at each of the participating laboratories, in particular, Francis Chan, MPH, and Susan Girard, BS, at Clinical Labs Hawaii; Ronald Fox, BS, Clyde Kanemoto, BS, and Raymond Yeung, MPH, at Diagnostic Laboratory Services/Accupath; and Les Chock, MS, Phuong Kamimura, BS, Susana Sum, BS, and Robert Young, BS, at Kaiser Permanente-Hawaii Laboratories. Our contractor, Eclipsys Corporation, deserves special thanks for assisting with the development of the extraction programs and electronic data transfer systems used in the ECDRS.
Corresponding Author and Reprints: Paul Effler, MD, MPH, State Epidemiologist, Department of Health, 1250 Punchbowl St, Honolulu, HI 96813.
Author Affiliations: State of Hawaii Department of Health, Honolulu (Dr Effler and Mss Ching-Lee, Bogard, Ieong, and Nekomoto); and the Office of Surveillance, National Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Ga (Dr Jernigan).
REFERENCES
| |
1. Institute of Medicine. Emerging Infections: Microbial Threats to Health in the United States. Lederberg J, Shope RE, Oaks SC Jr, eds. Washington, DC: National Academy Press; 1992.
2. Chorba TL, Berkelman RL, Safford SK, Gibbs NP, Hull HF. Mandatory reporting of infectious diseases by clinicians. MMWR Morb Mortal Wkly Rep. 1990;39(RR-9):1-17.
3. Thacker SB, Berkelman RL, Stroup DF. The science of public health surveillance. J Public Health Policy. 1989;10:187-203.
FULL TEXT
| PUBMED
4. Graitcer PL, Burton AH. The Epidemiologic Surveillance Project: a computer-based system for disease surveillance. Am J Prev Med. 1987;3:123-127.
WEB OF SCIENCE
| PUBMED
5. Thacker SB, Berkelman RL. Public health surveillance in the United States. Epidemiol Rev. 1988;10:164-190.
FREE FULL TEXT
6. Thacker SB, Choi K, Brachman PS. The surveillance of infectious diseases. JAMA. 1983;249:1181-1185.
FREE FULL TEXT
7. Institute of Medicine and National Research Council. Chemical and Biological Terrorism: Research and Development to Improve Civilian Medical Response. Washington, DC: National Academy Press; 1999.
8. Godes JR, Hall WN, Dean AG, Morse CD. Laboratory-based disease surveillance: a survey of state laboratory directors. Minn Med. 1982;65:762-764.
WEB OF SCIENCE
| PUBMED
9. State of Hawaii Department of Health. Cooperative Agreement No. U50/CCU912395-01. August 31, 1995.
10. Rushworth RL, Bell SM, Rubin GL, Hunter RM, Ferson MJ. Improving surveillance of infectious diseases in New South Wales. Med J Aust. 1991;154:828-831.
WEB OF SCIENCE
| PUBMED
11. Schramm MM, Vogt RL, Mamolen M. The surveillance of communicable disease in Vermont: who reports? Public Health Rep. 1991;106:95-97.
WEB OF SCIENCE
| PUBMED
12. Centers for Disease Control and Prevention. Preventing Emerging Infectious Diseases: A Strategy for the 21st Century. Atlanta, Ga: US Dept of Health and Human Services; 1998.
13. The White House. Presidential Decision Directive NSTC-7. Available at: http://www.whitehouse.gov/WH/EOP/OSTP/NSTC/html/pdd7.html. Accessed October 11, 1999.
14. State of Hawaii Department of Health. Cooperative Agreement No. U50/CCU912395-03. May 20, 1998.
15. Centers for Disease Control and Prevention. Electronic reporting of laboratory data for public health: meeting report and recommendations, November 23, 1997. Available at: http://www.phppo.cdc.gov. Accessed October 12, 1999.
16. Stroup DF. Special analytic issues. In: Teutsch SM, Churchill RE, eds. Principles and Practice of Public Health Surveillance. New York, NY: Oxford University Press; 1994:143-145.
17. McDonald CJ, Overhage JM, Dexter P, Takesue BY, Dwyer DM. A framework for capturing clinical data sets from computerized sources. Ann Intern Med. 1997;127(8 pt 2):675-682.
18. Health Level Seven. Available at: http://www.mcis.duke.edu/standards/HL7/hl7.htm. Accessed October 11, 1999.
19. Laboratory observation identifier names and codes Available at: http://www.mcis.duke.edu/standards/HL7/termcode/loinclab/loinc.html. Accessed March 11, 1999.
20. Systemized nomenclature for medicine Available at: http://snomed.org/. Accessed March 11, 1999.
21. Council of State and Territorial Epidemiologists. TABLE 2.—Reporting Requirements for Health Care Providers and Laboratories Diseases and Conditions Under National Surveillance. Available at: http://www.cste.org/table202B.html. Accessed March 10, 1999.
22. National Science and Technology Council. Infectious Disease: A Global Health Threat. Washington, DC: National Science and Technology Council; 1995.
23. US General Accounting Office. Emerging Infectious Diseases: Consensus on Needed Laboratory Capacity Could Strengthen Surveillance. Washington, DC: US General Accounting Office; 1999. GAO/HEHS-99-26.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
RELATED ARTICLE
November 17, 1999
JAMA. 1999;282(19):1885-1886.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Clinical Case Registries: Simultaneous Local and National Disease Registries for Population Quality Management
Backus et al.
J. Am. Med. Inform. Assoc. 2009;16:775-783.
ABSTRACT
| FULL TEXT
Review Article: Communicable diseases surveillance lessons learned from developed and developing countries: Literature review
Sahal et al.
Scand J Public Health 2009;37:187-200.
ABSTRACT
Collecting And Sharing Data For Population Health: A New Paradigm
Diamond et al.
Health Aff (Millwood) 2009;28:454-466.
ABSTRACT
| FULL TEXT
Electronic Support for Public Health: Validated Case Finding and Reporting for Notifiable Diseases Using Electronic Medical Data
Lazarus et al.
J. Am. Med. Inform. Assoc. 2009;16:18-24.
ABSTRACT
| FULL TEXT
Comparison of tuberculosis surveillance systems in low-incidence industrialised countries
Mor et al.
Eur Respir J 2008;32:1616-1624.
ABSTRACT
| FULL TEXT
A Comparison of the Completeness and Timeliness of Automated Electronic Laboratory Reporting and Spontaneous Reporting of Notifiable Conditions
Overhage et al.
AJPH 2008;98:344-350.
ABSTRACT
| FULL TEXT
Benefits and Barriers to Electronic Laboratory Results Reporting for Notifiable Diseases: The New York City Department of Health and Mental Hygiene Experience
Nguyen et al.
AJPH 2007;97:S142-S145.
ABSTRACT
| FULL TEXT
Sexually Transmitted Diseases and Managed Care: An Inquiry and Review of Issues Affecting Service Delivery
Chorba et al.
American Journal of Medical Quality 2004;19:145-156.
ABSTRACT
Systematic Review: Surveillance Systems for Early Detection of Bioterrorism-Related Diseases
Bravata et al.
ANN INTERN MED 2004;140:910-922.
ABSTRACT
| FULL TEXT
Design of a National Retail Data Monitor for Public Health Surveillance
Wagner et al.
J Am Med Inform Assoc 2003;10:409-418.
ABSTRACT
| FULL TEXT
Design of a National Retail Data Monitor for Public Health Surveillance
Wagner et al.
J. Am. Med. Inform. Assoc. 2003;10:409-418.
ABSTRACT
| FULL TEXT
Completeness of Notifiable Infectious Disease Reporting in the United States: An Analytical Literature Review
Doyle et al.
Am J Epidemiol 2002;155:866-874.
ABSTRACT
| FULL TEXT
Reporting of Laboratory-Confirmed Chlamydial Infection and Gonorrhea by Providers Affiliated With Three Large Managed Care Organizations--United States, 1995-1999
JAMA 2002;287:1933-1934.
FULL TEXT
|
