Original Contribution
JAMA. 2008;299(1):70-78. doi: 10.1001/jama.2007.64

Trends in Opioid Prescribing by Race/Ethnicity for Patients Seeking Care in US Emergency Departments

  1. Mark J. Pletcher, MD, MPH;
  2. Stefan G. Kertesz, MD, MSc;
  3. Michael A. Kohn, MD, MPP;
  4. Ralph Gonzales, MD, MSPH
  1. Author Affiliations: Department of Epidemiology and Biostatistics (Drs Pletcher and Kohn) and Division of General Internal Medicine, Department of Medicine (Drs Pletcher and Gonzales), University of California, San Francisco; Division of Preventive Medicine, University of Alabama at Birmingham (Dr Kertesz); Deep South Center on Effectiveness, Veterans Affairs Medical Center, Birmingham, Alabama (Dr Kertesz); and Emergency Department, Mills-Peninsula Medical Center, Burlingame, California (Dr Kohn).
  1. Corresponding Author: Mark J. Pletcher, MD, MPH, Department of Epidemiology and Biostatistics, University of California, San Francisco, 185 Berry St, Ste 5700, San Francisco, CA 94107 (mpletcher{at}epi.ucsf.edu).

More author information

Abstract

Context  National quality improvement initiatives implemented in the late 1990s were followed by substantial increases in opioid prescribing in the United States, but it is unknown whether opioid prescribing for treatment of pain in the emergency department has increased and whether differences in opioid prescribing by race/ethnicity have decreased.

Objectives  To determine whether opioid prescribing in emergency departments has increased, whether non-Hispanic white patients are more likely to receive an opioid than other racial/ethnic groups, and whether differential prescribing by race/ethnicity has diminished since 2000.

Design and Setting  Pain-related visits to US emergency departments were identified using reason-for-visit and physician diagnosis codes from 13 years (1993-2005) of the National Hospital Ambulatory Medical Care Survey.

Main Outcome Measure  Prescription of an opioid analgesic.

Results  Pain-related visits accounted for 156 729 of 374 891 (42%) emergency department visits. Opioid prescribing for pain-related visits increased from 23% (95% confidence interval [CI], 21%-24%) in 1993 to 37% (95% CI, 34%-39%) in 2005 (P < .001 for trend), and this trend was more pronounced in 2001-2005 (P = .02). Over all years, white patients with pain were more likely to receive an opioid (31%) than black (23%), Hispanic (24%), or Asian/other patients (28%) (P < .001 for trend), and differences did not diminish over time (P = .44), with opioid prescribing rates of 40% for white patients and 32% for all other patients in 2005. Differential prescribing by race/ethnicity was evident for all types of pain visits, was more pronounced with increasing pain severity, and was detectable for long-bone fracture and nephrolithiasis as well as among children. Statistical adjustment for pain severity and other factors did not substantially attenuate these differences, with white patients remaining significantly more likely to receive an opioid prescription than black patients (adjusted odds ratio, 0.66; 95% CI, 0.62-0.70), Hispanic patients (0.67; 95% CI, 0.63-0.72), and Asian/other patients (0.79; 95% CI, 0.67-0.93).

Conclusion  Opioid prescribing for patients making a pain-related visit to the emergency department increased after national quality improvement initiatives in the late 1990s, but differences in opioid prescribing by race/ethnicity have not diminished.

Inadequately treated pain is a major public health problem in the United States1 and a particular problem in emergency departments.2 Patients usually present to the emergency department when other medical help is not accessible or when symptoms, often including pain, are most severe. Emergency department visits therefore represent high-risk encounters in which assessment and treatment of pain should receive careful attention.2

Racial and ethnic minority groups appear to be at particularly high risk of receiving inadequate treatment for pain in the emergency department. For example, Hispanics with long-bone fracture presenting to an emergency department in Los Angeles were twice as likely to receive no opioid analgesic compared with non-Hispanic whites3; 2 other single-center studies have found similar results,4-5 although other studies have not.6-10 Previous studies of the National Hospital Ambulatory Medical Care Survey (NHAMCS), a survey of US emergency department visits, have found national racial/ethnic differences in opioid prescribing for back pain and migraine (1997-1999)6 and in provision of sedation for children with long-bone fractures (1992-1998).11

In the 1990s, national attention focused on increasing awareness of the problem of inadequately treated pain.1 Major campaigns undertaken by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)12 and the Veterans Health Administration13 introduced standards for consistent monitoring and treatment of pain that have become important quality indicators. Administrative data collected by the Drug Enforcement Agency show marked increases every year since the mid 1990s in the total quantity of opioids prescribed in the United States.14-15

It is unclear whether these generalized increases in opioid prescribing have resulted in more prescribing in the emergency department or a reduction in differential prescribing by race/ethnicity. We used 13 years of national survey data on emergency department visits collected by NHAMCS from 1993-2005 to investigate whether opioid prescribing is increasing in US emergency departments for patients presenting with pain, whether non-Hispanic white patients are more likely to receive an opioid than other racial/ethnic groups, and whether this gap has narrowed in recent years.

METHODS

Study Design and Administration

The NHAMCS was designed by the National Center for Health Statistics and is administered by the US Census Bureau to measure utilization and provision of ambulatory care services at US hospitals. Using a 4-stage probability sample design, NHAMCS collected a nationally representative sample of all visits to emergency departments based in noninstitutional general and short-stay hospitals, excluding federal, military, and Veterans Administration hospitals.

Hospital staff members at sampled hospitals, monitored by NHAMCS field representatives, collected visit information during a randomly assigned 4-week reporting period each year (data from 1993-2005 are used in this analysis). Data were processed at a central facility and then checked manually and by computerized algorithm to ensure data consistency. A detailed description of the NHAMCS methodology is available from the National Center for Health Statistics.16 Our analysis of this publicly available data set was exempted from institutional review board review by the University of California, San Francisco, institutional review board.

Study Sample: Emergency Department Visits With Pain

Up to 3 “complaints, symptoms, or other reason(s) for visit” were abstracted as free text and then coded centrally by NHAMCS personnel using a standard reason for visit classification (RVC) system.16-17 Coding was subject to a 2-way 10% independent verification procedure for quality control, and illegible entries were reviewed and adjudicated centrally. We identified a set of pain-related RVC codes that included pain, soreness, discomfort, ache, cramps, spasms, burning, or stinging. We then classified these by the location of the pain. When more than 1 pain-related RVC code was used, we classified the pain visit according to the first code listed. We limited our primary analyses to pain-related visits, defined as those having at least 1 pain-related RVC code.

Secondary analyses were conducted for each of the following visit categories: visits in which the first RVC code was pain-related; those in which a pain-related RVC code was not specifically noted but in which an injury-related RVC code was present; those with a level of pain severity recorded (“none,” “mild,” “moderate,” or “severe,” recorded in survey years 1997-2000 and 2003-2005 only); and those in which a typically painful diagnosis (long-bone fracture or nephrolithiasis) was noted by the treating physician. Up to 3 diagnoses are recorded for each visit as free text and then coded centrally by the NHAMCS using the International Classification of Diseases, Ninth Revision (ICD-9). We used ICD-9 codes 812, 813, 821, and 823 to identify visits with long-bone fracture and code 592 for those with nephrolithiasis.

Primary Predictor: Race/Ethnicity

Race (white, black, Asian/Pacific Islander, Native American, other, multiple) and ethnicity (Hispanic or non-Hispanic) were determined, per NHAMCS instructions, according to “your hospital's usual practice or based on your knowledge of the patient or from information in the medical record.” To identify minority groups most likely to experience health care disparities, we used non-Hispanic white patients (whites) as the control group and compared them with non-Hispanic black patients (blacks), non-Hispanic Asians/Native Americans/other/multiple patients (Asians/others), and Hispanic patients of any race (Hispanics). Asian patients made 84% of the observed visits in the Asian/other category in NHAMCS.

Covariates

Patient age, sex, and insurance status (categories collapsed to private, Medicare, Medicaid, workers' compensation, or self/no charge/other/unknown) are recorded for each visit. Using physician diagnosis and RVC codes, we identified patients with alcohol withdrawal, intoxication, and abuse (ICD-9 codes 291.0, 291.3, 291.4, 291.81, 303.0, and 305.0; RVC codes 1145.0, 4518.1, and 5915.0); alcohol dependence (ICD-9 codes 291.1, 291.2, 291.5, 291.8, 291.82, 291.89, 291.9, and 303.9; RVC code 2320.0); drug withdrawal, intoxication, and abuse (ICD-9 codes 292.0, 292.1, 292.2, 292.81, and 305.2-305.7; RVC codes 1150.0, 4518.0, and 5910.0); drug dependence (ICD-9 codes 292.82-292.89, 292.9, and 304; RVC code 2321.0); and sickle cell disease (ICD-9 codes 282.5 and 282.6). Hospital characteristics, including region (Northeast, Midwest, South, and West), owner (nonprofit, governmental, and proprietary), and setting (urban and rural) were obtained by NHAMCS from a proprietary national hospital database (Verispan LLC, Yardley, Pennsylvania).

Outcome: Any Opioid Analgesic Prescribed

Up to 6 medications (8 since 2003) administered during the visit or prescribed at discharge were recorded for each visit. Medications were coded via an ambulatory care drug database coding system.18 Coding was subject to a 2-way 10% independent verification procedure for quality control, and illegible entries were reviewed and adjudicated centrally. We used generic codes and ingredient codes to identify whether any administered or prescribed medications included specific opioid ingredients or acetaminophen. We used National Drug Code categories to identify nonsteroidal anti-inflammatory drugs and other nonopioid analgesics (code 17xx for analgesics, code 1728 for nonsteroidal anti-inflammatory drugs). In 2005, NHAMCS collected information on timing of prescription for each medication (administered during visit, at discharge, or both).

Data Analysis

We used the weights, strata, and primary sampling unit design variables provided by NHAMCS for all analyses. Repeat visits by the same individual are not accounted for, because unique identifiers are not provided in NHAMCS. However, because data are drawn from only 4 weeks' duration at a given facility, repeated visits are likely to be relatively uncommon in the accrued data and to occur most often within the primary sampling units and thus should not introduce much additional nonindependence.

Distributions of covariates and proportions receiving opioids were compared across race/ethnicity among pain-related visits using analysis of variance (for age) or χ2 tests (all others). Logistic regression was used with time included as a linear predictor to test time trends; differences in time trends (by race/ethnicity and before/after initiation of the JCAHO guidelines) were tested by including interaction terms with the time variable. Multivariable logistic regression was used to adjust for time (modeled as a continuous linear predictor), participant and hospital characteristics, and pain severity (in a subanalysis of survey years 1997-2000 and 2003-2005 only, when pain severity data were collected). P values for interaction refer to a test of the null hypothesis that all interaction coefficients are jointly equal to zero. All analyses used survey weights and took into account the complex survey design using the svy package provided in Stata version 9.2 (StataCorp, College Station, Texas). P values less than .05 were considered statistically significant.

RESULTS

During the 13 survey years (1993-2005), NHAMCS collected data on 374 891 emergency department visits. A pain-related RVC code was recorded for 156 729 (42%) of these visits. Using survey weights and averaging over our study period, we estimated that 44 million (95% confidence interval [CI], 41-47 million) pain-related visits are made annually to US emergency departments, representing 42.6% (95% CI, 42.1%-43.0%) of all emergency department visits. In 36 million of these visits (81%), the first RVC code listed was pain-related.

Whites made 66% of all pain-related visits; blacks, 20%; Hispanics, 11%; and Asians/others, 2% (Table 1). Whites were older and more likely to have private insurance than nonwhites; blacks were more likely to be diagnosed with sickle cell trait or disease, although these diagnoses were rare. Other differences in visit and patient characteristics were not large; types and severity of pain symptoms, for example, were similar across race/ethnicity groups (Table 1).

Table 1. Characteristics of Pain-Related US Emergency Department Visits, 1993-2005, by Race/Ethnicity

An opioid analgesic was prescribed at 29% (95% CI, 28%-30%) of pain-related visits. This proportion increased during the study period, from 23% (95% CI, 21%-24%) in 1993 to 37% (34%-39%) in 2005 (P < .001 for trend) (Figure 1). This trend was more pronounced starting in 2001, the year that JCAHO and the Veterans Health Administration initiated pain-related quality improvement efforts (P = .02 for interaction).

Figure 1. Percentage of Emergency Department Pain-Related Visits at Which an Opioid Was Prescribed, by Race/Ethnicity and Survey Year, NHAMCS 1993-2005

Opioid prescribing is increasing in all race/ethnic groups (P < .001), but the gap between whites and nonwhite groups has not decreased (P = .44 for interaction between time and race/ethnicity). NHAMCS indicates National Hospital Ambulatory Medical Care Survey. Error bars indicate 95% confidence intervals.

Despite this time trend, we found no evidence that differential opioid prescribing by race/ethnicity diminished over time (Figure 1). Averaged over the 13 survey years, opioid prescribing was more likely for pain-related visits made by whites (31%; 95% CI, 31%-32%) than by blacks (23%; 95% CI, 22%-24%), Hispanics (24%; 95% CI, 23%-26%), or Asians/others (28%; 95% CI, 26%-30%), and we found no evidence of an interaction between the time trend and race/ethnicity during the study period (P = .44). In 2005, opioid prescribing rates were 40% (95% CI, 37%-42%) in whites and 32% (95% CI, 29%-34%) in all others. Opioid prescribing was associated with pain severity, and differences were present for all levels of severity both before and after initiation of national pain-related quality improvement efforts (Figure 2).

Figure 2. Percentage of Emergency Department Visits at Which an Opioid Was Prescribed, by Pain Severity and Period, NHAMCS 1997-2000 and 2003-2005

Whites were more likely to receive an opioid prescription than nonwhites at every level of pain severity, both before (1997-2000) and after (2003-2005) national pain-related quality improvement efforts (P < .001 for all comparisons). There were no significant interactions between race/ethnicity and period for any level of pain (P > .20 for interaction in every subgroup of pain severity). Note that pain severity measurements are not available for survey years 1993-1996 or 2001-2002. NHAMCS indicates National Hospital Ambulatory Medical Care Survey. Error bars indicate 95% confidence intervals.

Differential opioid prescribing was consistently present across different types of pain, across different levels of pain severity, for visits in which pain was the first or second/third reason for visit, and for 2 specific painful diagnoses, long-bone fracture and nephrolithiasis (Table 2). Differences in prescribing between whites and nonwhites were larger as pain severity increased (P = .01 for interaction) and were particularly pronounced for patients with back pain (48% vs 36%, respectively), headache (35% vs 24%), abdominal pain (32% vs 22%), and other pain (40% vs 28%). Blacks were prescribed opioids at lower rates than any other race/ethnicity group for almost every type of pain visit. Prescribing rates were high for patients with a diagnosis of sickle cell disease and trended upward over time along with general opioid prescribing for pain (P = .05 for trend in all, P = .09 for trend in blacks), but the rarity of the diagnosis did not allow for detection of differential prescribing by race/ethnicity.

Table 2. Percentage of Emergency Department Visits at Which an Opioid Was Prescribed, by Race/Ethnicity and Type of Pain, Severity, and Physician Diagnosis, NHAMCS 1993-2005

Nonopioid analgesia alone (without opioids) was prescribed more frequently for nonwhites than whites (32% vs 26%), such that overall prescribing rates for any analgesic were not much different (Table 3). Differential opioid prescribing was accounted for primarily by a combination of higher prescribing of hydrocodone (12% vs 9%) and Schedule II opioids (14% vs 10%) but appeared to be present for all commonly prescribed opioids (with the exception of codeine) and for opioids administered during the visit and prescribed at discharge.

Table 3. Types of Analgesia Provided to Patients Presenting to the Emergency Department With Pain, by Race/Ethnicity, NHAMCS 1993-2005

Differences in opioid prescribing by race/ethnicity remained large and statistically significant after adjusting for patient, visit, and hospital characteristics (Table 4). In the subset of survey years in which pain severity was measured, additional adjustment for pain severity did not substantially attenuate observed patterns. In comparison with whites, opioid prescribing rates were particularly low among black and Hispanic children, blacks in government-owned (but nonfederal) hospitals and who self-pay, Asians/others with Medicare, and all nonwhites in hospitals located in the Northeast. There was little evidence of differential prescribing by sex (odds ratio for men vs women, 0.98; 95% CI, 0.95-1.00) and no evidence that differential prescribing by race/ethnicity differed by sex (P = .16 for interaction).

Table 4. Odds Ratios for Receiving Opioids During a Pain-Related Visit, by Race/Ethnicity, Overall and in Subgroups

COMMENT

In this analysis of national trends in opioid prescribing, we found that opioid prescribing rates for patients presenting to the emergency department with a complaint of pain have increased markedly in recent years, especially since implementation of national initiatives to improve pain-related care. Despite this trend of increasing opioid prescribing, we found no evidence that racial/ethnic differences in opioid prescribing have narrowed over time. In 2005, the absolute difference in prescribing rates was 8%, suggesting that of 12 or 13 patients of minority race/ethnicity presenting to an emergency department with pain, 1 fewer would receive an opioid analgesic than if all 12 or 13 patients were white. This difference was not attenuated by statistical adjustment for measurable confounders and was evident across different types of patients, pain complaints, pain severity and diagnoses, all types of hospitals (with some variation by hospital setting), and among children.

The increases in emergency department opioid prescribing we observed are consistent with national data that show a marked overall increase since the mid 1990s in the quantity of opioids prescribed annually in the United States.14-15 Our results show that this change is evident in emergency departments and is attributable, at least in part, to an increase in the likelihood of being prescribed an opioid analgesic and not just the quantity prescribed per prescription. We also found a significant increase in this trend starting in 2001 that coincides with national efforts by JCAHO12 to generally improve the quality of pain control in the United States, suggesting that the JCAHO initiative may have affected physician behavior.

The JCAHO initiative did not specifically target disparities in pain management, and as shown in a previous analysis of quality improvement for hemodialysis,19 improvements in quality do not necessarily eliminate disparities. The presence of differential opioid prescribing by race/ethnicity throughout the study period suggests that disparities in pain management persist in the emergency department. Most previous studies have shown evidence of opioid prescribing differences by race/ethnicity for a variety of types of pain and in a variety of settings, including pain from long-bone fracture,3-4,11 musculoskeletal pain,5 back pain,6 and migraine,6 although some have found no evidence of differences in opioid prescribing for long-bone fracture,6-10 for patients in a pediatric trauma registry,20 or for burn patients.21 Most of these prior studies represent single-center analyses that could reflect local prescribing cultures,3-5,7-9,20 and all but 1 study were conducted prior to the JCAHO initiative; the 1 recent study examines prescribing for African Americans at a single institution in Alabama.5 No prior analyses of national data have evaluated differential opioid prescribing following the JCAHO initiative. The present report also includes a wider range of patient visit types than previous analyses, classification of visits using physician diagnoses as well as the patient-centered RVC codes, and a longer study period that allows for examination of time trends.

A limitation of our analysis and others using NHAMCS is that the survey offers limited clinical details about each patient encounter. We do not know whether the patient requested pain medicine or what quantity of opioids was prescribed for each patient, and we do not have systematic information regarding drug and alcohol problems. On the other hand, we do have a measure of pain severity for 7 of the survey years (4 years before the JCAHO initiative and 3 years after). This measure appears to capture important information about pain severity (it is highly associated with an opioid prescription), and differential prescribing is evident within each level of pain severity. Another limitation is that race/ethnicity was determined by someone other than the patient. This may have led to some patients being categorized differently than they would have categorized themselves. However, in the context of studying disparities, the perception of patients' race/ethnicity may be more important than the patients' self-classification.

Important strengths of our study include the comprehensive examination of all types of pain-related visits, the ability to analyze trends over an important 13-year period during which national changes in policy occurred, and the nature of the sampling scheme, which allows our results to be generalized to all emergency department visits made in the United States during the study period.

It is unlikely that the differential prescribing by race/ethnicity we observed represents an appropriate pattern of care. Even though minorities report more often relying on the emergency department as a “usual source of care,”22 we found no evidence that pain severity was lower for nonwhites than it was for whites making a visit to the emergency department for pain. Residual confounding, in this case by residual differences in the severity or types of visits made by minorities, may still exist in this observational study. However, the size of the associations and the consistency with which we observed them across different types of pain visits and patients and despite adjustment for pain severity is evidence that physicians actually prescribe differently to whites than to nonwhites. Differential prevalence of alcohol and drug problems, though measured imperfectly in this study, does not appear to explain prescribing differences and would be unlikely to have been influential among persons younger than 12 years, in whom the racial/ethnic difference was especially pronounced.

It is conceivable that these prescribing differences represent a degree of overprescribing among white patients. Prescription opioid abuse has increased markedly during the last decade,23-25 and the emergency department is one setting that drug-seeking patients use to obtain opioid analgesics to sell or abuse.26 Emergency department physicians may be less likely to detect signs of abuse in white patients than in nonwhite patients, even though there is evidence that nonmedical use of prescription opioid analgesics is more common in whites.23, 27 Overprescribing is unlikely to explain the prescribing differentials for children younger than 12 years, patients with moderate to severe pain, or those with nephrolithiasis and long-bone fracture, 2 specific diagnoses that are consistently painful.

A more plausible explanation for our findings invokes a true disparity in prescribing, with differential undertreatment of pain in minority patients. There is substantial physician-level variability in the likelihood to prescribe opioids,28-29 and this variability may partially result from racial/ethnic bias. Causes of disparities in medical care, however, are complex, and simple racial/ethnic bias is unlikely to fully explain the problem.30 Race and ethnicity influence all aspects of the therapeutic relationship,31 including how (or whether) patients articulate painful symptoms to their physicians, what kinds of treatment are requested, and how physicians and allied health staff interpret and respond to those symptoms. While surveys of physicians do not show racial/ethnic differences in self-reported willingness to prescribe opioids for hypothetical scenarios32 or ethnicity bias when prompted to assess pain,33 physicians may assess the likelihood that a patient is exaggerating his or her pain symptoms to obtain pain medications for nonmedical purposes differently by ethnicity, though not always in expected directions.34

Also, white patients may be more likely to expect resolution of their pain and to request opioid analgesics on this basis, although at least 1 study has shown equal expectations of pain control comparing Hispanics with non-Hispanic whites.35 In the context of cancer-related pain, minority patients are less likely to perceive control over pain and its treatment,36 and interventions to improve patients' understanding of pain and to help them know how to communicate their pain in clinical encounters have improved pain control and reduced disparities between whites and minorities with cancer.37 Similar interactions of patient and physician assertiveness, self-efficacy, and expectations could help to explain racial/ethnic differences in opioid prescribing for emergency department patients with pain.

Our results suggest that new strategies are needed to understand and improve the quality and equity of management of acute pain in the United States. Future initiatives should continue to monitor pain management quality indicators and processes of care38 that may contribute to inadequate care, to educate physicians about the importance of adequate pain control, and to promote cultural competence within individual physicians.39 It is likely, however, that eliminating disparities in pain control will also require nonphysician interventions such as patient-targeted self-efficacy education,37 nurse-initiated pain-treatment protocols,40-42 and other system-level changes30 to facilitate equitable, systematic, and consistent alleviation of pain in emergency department patients.

AUTHOR INFORMATION

  1. Author Affiliations: Department of Epidemiology and Biostatistics (Drs Pletcher and Kohn) and Division of General Internal Medicine, Department of Medicine (Drs Pletcher and Gonzales), University of California, San Francisco; Division of Preventive Medicine, University of Alabama at Birmingham (Dr Kertesz); Deep South Center on Effectiveness, Veterans Affairs Medical Center, Birmingham, Alabama (Dr Kertesz); and Emergency Department, Mills-Peninsula Medical Center, Burlingame, California (Dr Kohn).

Corresponding Author: Mark J. Pletcher, MD, MPH, Department of Epidemiology and Biostatistics, University of California, San Francisco, 185 Berry St, Ste 5700, San Francisco, CA 94107 (mpletcher{at}epi.ucsf.edu).

Author Contributions: Dr Pletcher had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Pletcher, Gonzales.

Analysis and interpretation of data: Pletcher, Kertesz, Kohn, Gonzales.

Drafting of the manuscript: Pletcher, Kertesz.

Critical revision of the manuscript for important intellectual content: Kertesz, Kohn, Gonzales.

Statistical analysis: Pletcher, Kertesz.

Study supervision: Gonzales.

Financial Disclosures: None reported.

Funding/Support: This study was funded by Agency for Healthcare Research and Quality grant R03 HS016238 (Dr Pletcher and Dr Gonzales) and National Institute on Drug Abuse grant K23DA015487 (Dr Kertesz).

Role of the Sponsors: The Agency for Healthcare Research and Quality and the National Institute on Drug Abuse had no role in design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

REFERENCES

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.

Related articles

« Previous | Next Article »Table of Contents

More in JAMA & Archives Journals