 |
|
Validation of the Framingham Coronary Heart Disease Prediction Scores
Results of a Multiple Ethnic Groups Investigation
Ralph B. D'Agostino, Sr, PhD;
Scott Grundy, MD,PhD;
Lisa M. Sullivan, PhD;
Peter Wilson, MD;
for the CHD Risk Prediction Group
JAMA. 2001;286:180-187.
ABSTRACT
| |
Context The Framingham Heart Study produced sex-specific coronary heart disease (CHD) prediction functions for assessing risk of developing incident CHD in a white middle-class population. Concern exists regarding whether these functions can be generalized to other populations.
Objective To test the validity and transportability of the Framingham CHD prediction functions per a National Heart, Lung, and Blood Institute workshop organized for this purpose.
Design, Setting, and Subjects Sex-specific CHD functions were derived from Framingham data for prediction of coronary death and myocardial infarction. These functions were applied to 6 prospectively studied, ethnically diverse cohorts (n = 23 424), including whites, blacks, Native Americans, Japanese American men, and Hispanic men: the Atherosclerosis Risk in Communities Study (1987-1988), Physicians' Health Study (1982), Honolulu Heart Program (1980-1982), Puerto Rico Heart Health Program (1965-1968), Strong Heart Study (1989-1991), and Cardiovascular Health Study (1989-1990).
Main Outcome Measures The performance, or ability to accurately predict CHD risk, of the Framingham functions compared with the performance of risk functions developed specifically from the individual cohorts' data. Comparisons included evaluation of the equality of relative risks for standard CHD risk factors, discrimination, and calibration.
Results For white men and women and for black men and women the Framingham functions performed reasonably well for prediction of CHD events within 5 years of follow-up. Among Japanese American and Hispanic men and Native American women, the Framingham functions systematically overestimated the risk of 5-year CHD events. After recalibration, taking into account different prevalences of risk factors and underlying rates of developing CHD, the Framingham functions worked well in these populations.
Conclusions The sex-specific Framingham CHD prediction functions perform well among whites and blacks in different settings and can be applied to other ethnic groups after recalibration for differing prevalences of risk factors and underlying rates of CHD events.
INTRODUCTION
The Framingham Heart Study has developed mathematical functions for predicting risk of clinical coronary heart disease (CHD) events.1-5 These are derived multivariable mathematical functions that assign weights to major CHD risk factors such as sex, age, blood pressure, total cholesterol (TC), low-density lipoprotein cholesterol (LDL-C), high-density lipoprotein cholesterol (HDL-C), smoking behavior, and diabetes status. For a person free of cardiovascular disease, his/her CHD risk factors are entered into the function to produce a probability estimate of developing CHD within a certain time period (eg, the next 5 years). Recently, Framingham investigators developed a simplified model that incorporates blood pressure and cholesterol categories proposed by the Fifth Joint National Committee on Hypertension (JNC-V) and the National Cholesterol Education Program, Adult Treatment Panel II (NCEP-ATP II).5-8
The Framingham functions were developed to assess the relative importance of CHD risk factors and to quantify the absolute level of CHD risk for individual patients. The report of the third adult treatment panel (NCEP-ATP III) endorses knowledge of absolute CHD risk as a means of identifying those patients who are likely to benefit from aggressive primary prevention strategies and as a tool motivating patients to comply with them.9
The Framingham Heart Study consists of white middle-class individuals. Concern exists as to the generalizability of its CHD risk function to populations such as other whites, blacks, Asian Americans, Hispanics, and Native Americans. In January 1999 the National Heart, Lung, and Blood Institute convened a CHD Prediction Workshop to evaluate the performance of Framingham functions in non-Framingham populations.10
METHODS
Framingham Models
Sex-specific Framingham CHD risk functions were derived from 2439 men and 2812 women, 30 to 74 years of age, who were free of cardiovascular disease (CVD) at the time of their Framingham Heart Study examinations in 1971 to 1974. Participants attended either the 11th examination of the original Framingham Cohort4-5 or the initial examination of the Framingham Offspring Study.11 Coronary heart disease risk factors were routinely and systematically evaluated during these examinations as described in detail elsewhere.5 Twelve-year follow-up was obtained for the development of "hard" CHD events, defined as coronary death or myocardial infarction. Sex-specific Cox proportional hazards regression functions were computed that relate JNC-V blood pressure and NCEP-ATP II cholesterol categories, along with age, current smoking, and presence of diabetes to the occurrence of hard CHD events.
Non-Framingham Cohorts
Six non-Framingham cohorts were identified for evaluation.12-17 Criteria for selection were similar age range, systematic measurement of CHD risk factors, and adequate length of follow-up for hard CHD events. The selected cohorts were participants in the Atherosclerosis Risk in Communities Study (ARIC, 1987-1988), the Physicians' Health Study (PHS, 1982), the Honolulu Heart Program (HHP, 1980-1982), the Puerto Rico Heart Health Program (PR, 1965-1968), the Strong Heart Study (SHS, 1989-1991), and the Cardiovascular Health Study (CHS, 1989-1990). The PHS is a prospective, nested case-control study with 1-to-4 matching of cases to controls for age and smoking.
For each cohort, sex-specific Cox regression functions were derived using the same variables as in the Framingham functions but using data from the individual cohorts. We call these the cohorts' "own" functions. They represent the best possible Cox prediction functions for each cohort based on specific prevalences of risk factors and CHD event rates. For the ARIC study, which includes white and black subjects, the Cox regression functions were sex- and race-specific. The CHS cohort included subjects 65 to 88 years old. We used only CHS subjects aged 65 to 74 years for the CHS' own functions.
Statistical Analysis
All analyses were sex- and race-specific. The performance of the Framingham prediction functions among the non-Framingham cohorts was assessed according to 3 evaluations: equality of regression coefficients (relative risk [RR] comparison), discrimination, and calibration.
Relative Risk Comparison
For each risk factor, Cox proportional hazards modeling yielded regression coefficients for the Framingham and non-Framingham cohorts. To compare these coefficients a test statistic z was calculated, where z = (b[F] - b[O])/SE, and where b(F) and b(O) are, respectively, the regression coefficients of the Framingham and the other cohort's model, while SE is the standard error of the difference in the coefficients. This was computed as the square root of the sum of the squares of the SEs for the 2 coefficients. Because the RR of a variable is computed by exponentiating its regression coefficient, the z statistic tests the equality of RRs between Framingham and non-Framingham cohorts.
Discrimination
Discrimination is the ability of a prediction model to separate those who experience hard CHD events from those who do not. We quantified this by calculating the c statistic, analogous to the area under a receiver operating characteristic (ROC) curve18-20; this value represents an estimate of the probability that a model assigns a higher risk to those who develop CHD within a 5-year follow-up than to those who do not.18-19 For each non-Framingham cohort 2 c statistics were computed, one applying the Framingham function to the cohort and the other from the cohort's own prediction function. These were compared using a test developed by Nam.20
Calibration
Calibration measures how closely predicted outcomes agree with actual outcomes. For this we used a version of the Hosmer-Lemeshow  2 statistic.19-20 For each non-Framingham cohort, the Framingham function's predicted risks were used to divide subjects into deciles of predicted risk for experiencing a hard CHD event within 5 years. Plots were constructed showing predicted and actual event rates for each decile. A 2 statistic was calculated to compare the differences between predicted and actual event rates; small values indicated good calibration. Values exceeding 20 indicate significant lack of calibration (P<.01). For further evaluation of calibration, we compared this 2 statistic with one derived from each cohort's own prediction function. All statistical analyses were performed in SAS version 6.12 (SAS Institute, Cary, NC). Because the PHS is a case-control study it is not suitable for calibration comparisons.
Recalibration
When there is a systematic overestimation or underestimation of risk, transporting a prediction function from one setting to another requires a process known as recalibration. The Framingham Cox regression models have the form S0(t)exp(f[x,M]) where f(x,M) = b1(x1- M1) + . . . + bp(xp- Mp). Here b1, . . . ,bp are the regression coefficients (logs of the RRs), x1, . . . ,xp represent an individual's risk factors, and M1, . . . ,Mp are the means of the risk factors of the Framingham cohort. S0(t) is the Framingham average incidence rate at time t or, more precisely, the survival rate at the mean values of the risk factors. With recalibration, the Framingham mean values of the risk factors (M1, . . . ,Mp) are replaced by the mean values of the risk factors from a non-Framingham cohort, while the Framingham average incidence rate S0(t) is replaced by the cohort's own average incidence rate. We used Kaplan-Meier estimates to determine average incidence rates.20-21 It is important to note that recalibration does not affect RR comparisons or discrimination evaluations.
RESULTS
Baseline Characteristics and Framingham Cohort Coefficients
The Framingham Study cohort consisted of 2439 men and 2812 women free of cardiovascular disease. The 5- and 10-year hard CHD event rates were 3.7% and 8.0% for men and 1.4% and 2.8% for women.
Table 1 shows the Cox regression coefficients for the sex-specific Framingham regression models. Table 2 contains the racial compositions, sample sizes, age ranges, mean ages, risk factor distributions, and 5-year incidence rates for the Framingham and non-Framingham cohorts.
|
|
|
|
Table 1. Framingham Functions (Cox Regression Coefficients) for Hard CHD Events (Coronary Death or MI)*
|
|
|
|
|
|
|
Table 2. Description of Studies Used in Evaluation*
|
|
|
Relative Risk Comparisons
Table 3 and Table 4 contain the RRs of the CHD risk factors for each cohort's sex- and race-specific regression model. First, we considered each function separately. Among men most risk factors had statistically significant coefficients, whereas among women a number of coefficients were not statistically significant, presumably because of low event rates (eg, for ARIC black women there were only 38 events among 2333 subjects). Nonetheless, within risk factor categories, trends were significant. For example, except for the SHS Native Americans, for all cohorts the risk for hard CHD events increased as blood pressure went from optimal to stage II-IV hypertension, as TC increased, and as HDL-C decreased (P<.01 for all). In the SHS there were some unexpected but not statistically significant elevated risks in high HDL-C groups.
|
|
|
|
Table 3. Relative Risks for CHD Risk Factors: Men by Study*
|
|
|
|
|
|
|
Table 4. Relative Risks for CHD Risk Factors: Women by Study*
|
|
|
Among men, there were no significant differences between Framingham RRs and those of ARIC white and black men. There were differences in RRs for smoking and age in the PHS cohort, which may reflect the matching scheme that was used in that study. Among HHP Japanese American men and PR Hispanic men, RRs were lower for optimal blood pressure. Smoking was associated with a lower RR among HHP Japanese American men, while diabetes and TC 280 mg/dL (7.25 mmol/L) or higher were associated with much higher RRs among SHS Native American men. Also, in this cohort HDL-C in the range of 50 to 59 mg/dL (1.30-1.53 mmol/L) had an unexpected elevated risk and stage I hypertension had an unexpected low risk, both resulting in significant differences from the Framingham function. Among the more elderly CHS white men, cholesterol abnormalities were associated with a lower RR.
Among women, there were no differences between Framingham RRs and those of white ARIC or CHS women. Black women in the ARIC cohort had higher RRs for high normal blood pressure and stage II-IV hypertension. In the SHS there were significant differences for diabetes and smoking. Also, HDL-C greater than or equal to 60 mg/dL (1.55 mmol/L) carried an elevated risk in the SHS, resulting in a significantly different RR than that of the Framingham function.
Discrimination
Table 5 contains the c statistics for both men and women. The "FHS" row refers to the discrimination achieved by applying the Framingham prediction functions to the non-Framingham cohorts, while the "Best Cox" row contains the c statistics resulting from the cohort's own Cox regression function. Since they are based on the same cohorts for which the scores were developed, the latter c statistics are overestimates. For non-Framingham white men (ARIC, PHS, and CHS) and women (ARIC and CHS), the Framingham functions discriminated well, almost always achieving the same discrimination as best Cox functions. Overall, within sampling fluctuations the Framingham functions discriminated nearly as well as the best Cox functions of the non-Framingham cohorts, with the exception of the SHS Native Americans.
|
|
|
|
Table 5. Summary of Discrimination and Calibration Evaluations*
|
|
|
Calibration
Table 5 also contains the 2 statistics for evaluation of the calibration of the Framingham prediction functions applied to non-Framingham cohorts. For white men and women, including the more elderly subjects in the CHS cohort, both the Framingham functions ("Unadjusted" row) and the individual cohort's own functions ("Best Cox" row) showed a statistically acceptable calibration. Figure 1 contains calibration plots for white and black men and women from the ARIC study. In general, actual CHD event rates were similar to event rates predicted by Framingham functions among white and black men and women.
|
|
|
|
Figure 1. Five-Year Predictions for Hard CHD Events: Performance Measures for ARIC Men and Women
X-axes refer to decile of predicted risk based on the Framingham Heart Study function. ARIC indicates Atherosclerosis Risk in Communities Study. Hard CHD events were coronary death or myocardial infarction.
|
|
|
For the HHP Japanese American men and the PR Hispanic men, the calibration 2 statistics of 66.0 and 142.0, respectively, indicate poor calibration (Table 5, "Unadjusted" row and Figure 2A and B, "Unadjusted" panels). The Framingham prediction function systematically overestimated risk in both cohorts, in which the overall CHD event rates were substantially lower. Model recalibration using the non-Framingham cohorts' mean values for risk factors and CHD incidence rates substantially improved the performance of the Framingham prediction functions (Figure 2, "Adjusted" panels, and Table 5). In the SHS, calibration was good for men (2 = 10.6), but less good for women (2 = 22.7). Recalibration resulted in improved performance of the Framingham functions (Figure 2 and Table 5).
|
|
|
|
Figure 2. Five-Year Predictions for Hard CHD Events: Performance Measures for HHP, PR, and SHS Subjects
X-axes refer to decile of predicted risk based on the Framingham Heart Study function. HHP indicates Honolulu Heart Program; PR, Puerto Rico Heart Health Program; and SHS, Strong Heart Study. Hard CHD events were coronary death or myocardial infarction.
|
|
|
COMMENT
The Framingham CHD prediction functions were developed to help clinicians estimate the absolute risk of any individual patient developing clinically manifest disease. We sought to demonstrate the external validity of the Framingham functions by examining their performance in 6 different well-described population-based cohorts that reflect a wide range of ethnic diversity.
As shown in Table 3 and Table 4, RRs for major CHD risk factors were remarkably similar to those derived from the Framingham Heart Study cohort among white men and women and black men in the ARIC cohort. Among black women in the ARIC cohort, RRs for elevated blood pressure were somewhat higher. In the cohorts that were made up of other ethnic groups, however, we did note some differences in RRs. For example, smoking was associated with a much lower RR in HHP Japanese American men for reasons that are not clear. In the CHS cohort, cholesterol abnormalities and smoking had lower RRs, possibly due to age interactions. In the SHS Native American cohort, there were RR differences for cholesterol abnormalities and diabetes. Some cholesterol differences are unexplained, with high HDL-C levels carrying an increased risk in the SHS cohort. Because the prevalence of diabetes among Native Americans is quite high, it is possible that the different RRs we observed may be due to interactions with other risk factors and with other factors unique to diabetes, such as albuminuria, that were not considered in our analyses.
The ability of the Framingham prediction functions to discriminate between subjects who developed clinical CHD and those who did not was reasonably good for most of the non-Framingham cohorts (Table 5). Among ARIC black women, the Framingham c statistic was numerically, but not significantly, lower than that derived from the model based on that same cohort's data. Since this ARIC c statistic is based on the same data with which its function was developed, it is an overestimate and the difference may relate to this. It may also be due to the small number of CHD events. The Framingham c statistic for CHS men was also numerically, but not significantly, lower than the CHS' own function c statistic. The difference may relate to the overestimate of the CHS cohort's own function c statistic. The reasons why both the Framingham and CHS cohort's own function c statistics are low may be a consequence of the relatively small sample size and the narrow age distribution. The c statistics were appreciably decreased for SHS Native Americans. Why discrimination was worse for Native Americans compared with that for white and black men and women of the ARIC cohort, PR Hispanic men, and HHP Japanese American men is not clear. It is possible that the markedly different RR estimates for cholesterol and diabetes among the SHS Native Americans may have adversely affected the ability of the Framingham prediction function to discriminate CHD risk.
In our model calibration analyses, we found reasonably good agreement between predicted and actual CHD event rates for all of the non-Framingham cohorts studied (Figure 1 and Figure 2, and Table 5) except for HHP Japanese American men, PR Hispanic men, and SHS Native American women. In these groups, the Framingham prediction functions systematically overestimated CHD risk (Figure 2A, B, and D). This overestimation was corrected by using a process of recalibration. In order to apply this to other such populations, it would be necessary to obtain cross-sectional data on risk factor prevalences as well as population data on CHD event rates over time.
Authors of treatment guidelines have recognized the need to have an accurate and reliable multivariable-based estimate of absolute CHD risk in order to best identify those most in need of aggressive preventive treatment.7-9 Thus, the recently released NCEP-ATP III guidelines specifically recommend that the level of treatment should relate to the level of CHD risk.9 They specifically incorporate Framingham prediction functions to aid clinicians and patients in determining optimal strategies. Simple charts can be used to aid in this activity.5, 9
In order for multivariable risk assessment and treatment guidelines to have optimal use and acceptability, clinicians need to be confident that absolute risk prediction functions can be transported to other settings beyond where they were originally developed. We have demonstrated that the FHS prediction functions work reasonably well among white and black men and women. When applied to Japanese American and Hispanic men and Native American women, recalibration was needed. Future work is needed to devise practical schemes by which clinicians can confidently apply the FHS prediction functions in these groups.
AUTHOR INFORMATION
Author Contributions: Study concept and design: D'Agostino, Grundy, Wilson.
Acquisition of data: D'Agostino, Grundy, Wilson.
Analysis and interpretation of data: D'Agostino, Grundy, Sullivan, Wilson.
Drafting of the manuscript: D'Agostino, Grundy.
Critical revision of the manuscript for important intellectual content: D'Agostino, Grundy, Sullivan, Wilson.
Statistical expertise: D'Agostino, Sullivan, Wilson.
Obtained funding: D'Agostino, Grundy.
Administrative, technical, or material support: D'Agostino, Grundy, Sullivan.
Study supervision: D'Agostino, Wilson.
Members of the CHD Risk Prediction Group: Boston University and the Framingham Study: Philip A. Wolf, MD, Daniel Levy, MD, Joseph Massaro, PhD, Byung-Ho Nam, PhD; CHD Risk Prediction Planning Committee (NIH): National Heart, Lung, and Blood Institute: James L. Cleeman, MD, Jeffrey A. Cutler, MD, Lawrence Friedman, MD, Edward Rocella, MD; CHD Risk Prediction Planning Committee (non-NIH): Scott M. Grundy, MD (chair), Ralph B. D'Agostino, Sr, PhD, Gregory Burke, MD, Lori Mosca, MD, Daniel Rader, MD, Peter W. F. Wilson, MD; Atherosclerosis Risk in Communities Study: Lloyd E. Chambless, PhD, David J. Couper, PhD; Physicians' Health Study: Meir J. Stampfer, MD, Jing Ma, PhD; Honolulu Heart Program: David Curb, MD, B. Rodrigues, MD, Robert Abbott, PhD; Puerto Rico Heart Health Program: Mario R. Gacia-Palmieri, MD, Paul Sorlie, PhD, Sean Coady, PhD; Strong Heart Study: Elisa Lee, PhD, Barbara Howard, MD; Cardiovascular Health Study: Richard Kronmal, PhD, Thomas Lumney, PhD.
Funding/Support: The Framingham Study work and analyses for the workshop carried out at Boston University were supported by a contract (N01-HC-380380) and funds from the National Heart, Lung, and Blood Institute of the National Institutes of Health.
Acknowledgment: We especially thank Claude Lenfant, MD, Director of the National Heart, Lung, and Blood Institute, for his support and organization of the Workshop on CHD Risk Assessment, January 1999, and to the cardiovascular studies (Atherosclerosis Risk in Community Study, Physicians' Health Study, Honolulu Health Project, Puerto Rico Heart Health Program, Strong Heart Study, and the Cardiovascular Health Study) who supported the workshop and this investigation by contributing their data and performing numerous analyses.
Corresponding Author and Reprints: Ralph B. D'Agostino, Sr, PhD, Department of Mathematics and Statistics, Boston University, 111 Cummington St, Boston, MA 02215 (e-mail: ralph{at}bu.edu).
Author Affiliations: Departments of Mathematics and Statistics (Drs D'Agostino and Sullivan) and Medicine (Dr Wilson), Boston University, Boston, Mass; the Framingham Study, Framingham, Mass (Drs D'Agostino, Sullivan, and Wilson); and Center for Human Nutrition, University of Texas Southwestern Medical Center, Dallas (Dr Grundy).
REFERENCES
| |
1. Coronary Risk Handbook: Estimating Risk of Coronary Heart Disease in Daily Practice. New York, NY: American Heart Association; 1973:1-35.
2. Kannel WB, McGee D, Gordon T. A general cardiovascular risk profile: the Framingham Study. Am J Cardiol. 1976;38:46-51.
FULL TEXT
|
ISI
| PUBMED
3. Gordon T, Kannel WB. Multiple risk functions for predicting coronary heart disease: the concept, accuracy, and application. Am Heart J. 1982;103:1031-1039.
FULL TEXT
|
ISI
| PUBMED
4. Anderson M, Wilson PW, Odell PM, Kannel WB. An updated coronary risk profile: a statement for health professionals. Circulation. 1991;83:356-362.
FREE FULL TEXT
5. Wilson PWF, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837-1847.
FREE FULL TEXT
6. The National Cholesterol Education Program Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel II). National Cholesterol Education Program, second report. Circulation. 1994;89:1333-1445.
PUBMED
7. The National Cholesterol Education Program Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel II). Summary of the second report of the NCEP Expert Panel (Adult Treatment Panel II). JAMA. 1993;269:3015-3023.
FREE FULL TEXT
8. Joint National Committee on Detection, Evaluation, and Treatment of High Blood Pressure (JNC-V). The fifth report of the Joint National Committee on Detection, Evaluation, and Treatment of High Blood Pressure (JNC-V). Arch Intern Med. 1993;153:154-183.
FREE FULL TEXT
9. National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Executive summary of the third report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). JAMA. 2001;285:2486-2497.
FREE FULL TEXT
10. Grundy SM, D'Agostino Sr RB, Mosca L, et al. Cardiovascular risk assessment based on US cohort studies: findings from a National Heart, Lung, and Blood Institute workshop. Circulation. 2001;104:1-6.
11. Kannel WB, Feinlieb M, McNamara PM, Garrison RJ, Castelli WP. An investigation of coronary heart disease in families: the Framingham Offspring Study. Am J Epidemiol. 1979;110:281-290.
FREE FULL TEXT
12. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol. 1989;129:687-702.
FREE FULL TEXT
13. Stampfer MJ, Sacks FM, Salvini S, Willett WC, Hennekens CH. A prospective study of cholesterol, apolipoproteins, and the risk of myocardial infarction. N Engl J Med. 1991;325:373-381.
ABSTRACT
14. Kagan A, Gordon T, Rhoads GG, Schiffman JC. Some factors related to coronary heart disease incidence in Honolulu Japanese men: the Honolulu Heart Study. Int J Epidemiol. 1975;4:271-279.
FREE FULL TEXT
15. Garcia-Palmieri MR, Costas R. Risk factors of coronary heart disease: a prospective epidemiologic study in Puerto Rico. In: Yu PH, Goodwin JF, eds. Progress in Cardiology. Vol 14. Philadelphia, Pa: Lea & Febiger; 1986:101-190.
16. Lee ET, Welty TK, Fabsitz R, et al. The Strong Heart Study: a study of cardiovascular disease in American Indians: design and methods. Am J Epidemiol. 1990;132:1141-1155.
FREE FULL TEXT
17. Fried LP, Borhani NO, Enright P, et al. The Cardiovascular Health Study: design and rationale. Ann Epidemiol. 1991 Feb;1(3):263-276.
18. Hanley JA, McNeill BJ. The measure and use of the area under the receiver operating characteristic (ROC) curve. Radiology. 1982;143:29-36.
FREE FULL TEXT
19. D'Agostino Sr RB, Griffith JL, Schmid CH, Terrin N. Measures for evaluating model performance. From: Annual Meeting of the American Statistical Association; Chicago, Ill, August 1996. In: American Statistical Association 1996 Proceedings of the Section on Biometrics. Alexandria, Va: American Statistical Association; 1997:253-258.
20. Nam B-H. Discrimination and Calibration in Survival Analysis [dissertation]. Boston, Mass: Boston University; 2000.
21. Colette D. Modeling Survival Data in Medical Research. London, England: Chapman & Hall; 1994.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati
What's this?
RELATED ARTICLE
July 11, 2001
JAMA. 2001;286(2):243-244.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
QRISK or Framingham for predicting cardiovascular risk?
Jackson et al.
BMJ 2009;339:b2673-b2673.
FULL TEXT
Predicting the 30-Year Risk of Cardiovascular Disease: The Framingham Heart Study
Pencina et al.
Circulation 2009;119:3078-3084.
ABSTRACT
| FULL TEXT
Impact of Adding a Single Allele in the 9p21 Locus to Traditional Risk Factors on Reclassification of Coronary Heart Disease Risk and Implications for Lipid-Modifying Therapy in the Atherosclerosis Risk in Communities Study
Brautbar et al.
Circ Cardiovasc Genet 2009;2:279-285.
ABSTRACT
| FULL TEXT
Evaluating risk for cardiovascular diseases--vain or value? How do different cardiovascular risk scores act in real life
Ketola et al.
Eur J Public Health 2009;0:ckp070v1-ckp070.
ABSTRACT
| FULL TEXT
Prediction Model for Prevalence and Incidence of Advanced Age-Related Macular Degeneration Based on Genetic, Demographic, and Environmental Variables
Seddon et al.
IOVS 2009;50:2044-2053.
ABSTRACT
| FULL TEXT
Effectiveness of Valsartan for Treatment of Hypertension: Patient Profiling and Hierarchical Modeling of Determinants and Outcomes (the PREVIEW Study)
Van der Niepen et al.
The Annals of Pharmacotherapy 2009;43:849-861.
ABSTRACT
| FULL TEXT
Framingham Stroke Risk Function in a Large Population-Based Cohort of Elderly People: The 3C Study
Bineau et al.
Stroke 2009;40:1564-1570.
ABSTRACT
| FULL TEXT
Lifestyle Interventions Reduce Coronary Heart Disease Risk: Results From the PREMIER Trial
Maruthur et al.
Circulation 2009;119:2026-2031.
ABSTRACT
| FULL TEXT
Impact of Whole-Body CT Screening on the Cost-effectiveness of CT Colonography
Hassan et al.
Radiology 2009;251:156-165.
ABSTRACT
| FULL TEXT
Lipoprotein Particle Profiles by Nuclear Magnetic Resonance Compared With Standard Lipids and Apolipoproteins in Predicting Incident Cardiovascular Disease in Women
Mora et al.
Circulation 2009;119:931-939.
ABSTRACT
| FULL TEXT
Impact of Computer-aided Detection on the Cost-effectiveness of CT Colonography
Regge et al.
Radiology 2009;250:488-497.
ABSTRACT
| FULL TEXT
Utility of the Seattle heart failure model in patients with advanced heart failure.
Kalogeropoulos et al.
J Am Coll Cardiol 2009;53:334-342.
ABSTRACT
| FULL TEXT
Prevalence and Progression of Subclinical Atherosclerosis in Younger Adults With Low Short-Term but High Lifetime Estimated Risk For Cardiovascular Disease: The Coronary Artery Risk Development in Young Adults Study and Multi-Ethnic Study of Atherosclerosis
Berry et al.
Circulation 2009;119:382-389.
ABSTRACT
| FULL TEXT
Cardiovascular Disease Risk Prediction With and Without Knowledge of Genetic Variation at Chromosome 9p21.3
Paynter et al.
ANN INTERN MED 2009;150:65-72.
ABSTRACT
| FULL TEXT
Progressing From Risk Factors to Omics
Wilson
Circ Cardiovasc Genet 2008;1:141-146.
FULL TEXT
Clinical Risk Factors, DNA Variants, and the Development of Type 2 Diabetes
Lyssenko et al.
NEJM 2008;359:2220-2232.
ABSTRACT
| FULL TEXT
Relation of N-terminal pro-brain natriuretic peptide levels and their prognostic power in chronic stable heart failure to obesity status
Frankenstein et al.
Eur Heart J 2008;29:2634-2640.
ABSTRACT
| FULL TEXT
C-Reactive Protein and Reclassification of Cardiovascular Risk in the Framingham Heart Study
Wilson et al.
Circ Cardiovasc Qual Outcomes 2008;1:92-97.
ABSTRACT
| FULL TEXT
Impact of High-Normal Blood Pressure on the Risk of Cardiovascular Disease in a Japanese Urban Cohort: The Suita Study
Kokubo et al.
Hypertension 2008;52:652-659.
ABSTRACT
| FULL TEXT
Hyponatremia and Mortality among Patients on the Liver-Transplant Waiting List
Kim et al.
NEJM 2008;359:1018-1026.
ABSTRACT
| FULL TEXT
Carotid Intima Media Thickness and Plaques Can Predict the Occurrence of Ischemic Cerebrovascular Events
Prati et al.
Stroke 2008;39:2470-2476.
ABSTRACT
| FULL TEXT
Marine-Derived n-3 Fatty Acids and Atherosclerosis in Japanese, Japanese-American, and White Men: A Cross-Sectional Study
Sekikawa et al.
J Am Coll Cardiol 2008;52:417-424.
ABSTRACT
| FULL TEXT
Cardiac Multidetector CT: Technical and Diagnostic Evaluation with Evidence-based Practice Techniques
Heffernan et al.
Radiology 2008;248:366-377.
ABSTRACT
| FULL TEXT
Ethnic Group Disparities in 10-Year Trends in Stroke Incidence and Vascular Risk Factors: The South London Stroke Register (SLSR)
Heuschmann et al.
Stroke 2008;39:2204-2210.
ABSTRACT
| FULL TEXT
Development and Validation of a Model for Predicting Emergency Admissions Over the Next Year (PEONY): A UK Historical Cohort Study
Donnan et al.
Arch Intern Med 2008;168:1416-1422.
ABSTRACT
| FULL TEXT
Ankle Brachial Index Combined With Framingham Risk Score to Predict Cardiovascular Events and Mortality: A Meta-analysis
Ankle Brachial Index Collaboration
JAMA 2008;300:197-208.
ABSTRACT
| FULL TEXT
Prediction of First Events of Coronary Heart Disease and Stroke With Consideration of Adiposity
Wilson et al.
Circulation 2008;118:124-130.
ABSTRACT
| FULL TEXT
Criteria for the Evaluation of Large Cohort Studies: An Application to the Nurses' Health Study
Colditz and Winn
JNCI J Natl Cancer Inst 2008;100:918-925.
ABSTRACT
| FULL TEXT
Global Coronary Heart Disease Risk Assessment of Individuals With the Metabolic Syndrome in the U.S.
Hoang et al.
Diabetes Care 2008;31:1405-1409.
ABSTRACT
| FULL TEXT
Beyond Established and Novel Risk Factors: Lifestyle Risk Factors for Cardiovascular Disease
Mozaffarian et al.
Circulation 2008;117:3031-3038.
FULL TEXT
Evaluation of the Framingham Risk Score in the European Prospective Investigation of Cancer-Norfolk Cohort--Invited Commentary
Pencina and D'Agostino
Arch Intern Med 2008;168:1216-1218.
FULL TEXT
Predicting values of lipids and white blood cell count for all-site cancer in type 2 diabetes
Yang et al.
Endocr Relat Cancer 2008;15:597-607.
ABSTRACT
| FULL TEXT
Coronary Calcium as a Predictor of Coronary Events in Four Racial or Ethnic Groups
Detrano et al.
NEJM 2008;358:1336-1345.
ABSTRACT
| FULL TEXT
Screening High-Risk Patients With Computed Tomography Angiography
Gottlieb and Lima
Circulation 2008;117:1318-1332.
FULL TEXT
Development and Validation of an All-Cause Mortality Risk Score in Type 2 Diabetes: The Hong Kong Diabetes Registry
Yang et al.
Arch Intern Med 2008;168:451-457.
ABSTRACT
| FULL TEXT
Using Clinical Factors and Mammographic Breast Density to Estimate Breast Cancer Risk: Development and Validation of a New Predictive Model
Tice et al.
ANN INTERN MED 2008;148:337-347.
ABSTRACT
| FULL TEXT
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D'Agostino et al.
Circulation 2008;117:743-753.
ABSTRACT
| FULL TEXT
Impact of Impaired Fasting Glucose on Cardiovascular Disease: The Framingham Heart Study
Levitzky et al.
J Am Coll Cardiol 2008;51:264-270.
ABSTRACT
| FULL TEXT
A Risk Score for Predicting Near-Term Incidence of Hypertension: The Framingham Heart Study
Parikh et al.
ANN INTERN MED 2008;148:102-110.
ABSTRACT
| FULL TEXT
Development and Validation of a Prognostic Index for Health Outcomes in Chronic Obstructive Pulmonary Disease
Briggs et al.
Arch Intern Med 2008;168:71-79.
ABSTRACT
| FULL TEXT
Cardiovascular risk prediction: are we there yet?
Jackson
Heart 2008;94:1-3.
FULL TEXT
Differences in Cardiovascular Disease Mortality Associated With Body Mass Between Black and White Persons
Abell et al.
Am. J. Public Health 2008;98:63-66.
ABSTRACT
| FULL TEXT
Coronary Heart Disease Risk Assessment by Traditional Risk Factors and Newer Subclinical Disease Imaging: Is a "One-Size-Fits-All" Approach the Best Option?
Preis and O'Donnell
Arch Intern Med 2007;167:2399-2401.
FULL TEXT
Projecting Individualized Absolute Invasive Breast Cancer Risk in African American Women
Gail et al.
JNCI J Natl Cancer Inst 2007;99:1782-1792.
ABSTRACT
| FULL TEXT
A coronary heart disease risk model for predicting the effect of potent antiretroviral therapy in HIV-1 infected men
May et al.
Int J Epidemiol 2007;36:1309-1318.
ABSTRACT
| FULL TEXT
External Prognostic Validations and Comparisons of Age- and Gender-Adjusted Exercise Capacity Predictions
Kim et al.
J Am Coll Cardiol 2007;50:1867-1875.
ABSTRACT
| FULL TEXT
Adherence with advice and prescriptions in SLE is mostly good, but better follow up is needed: A study with a questionnaire
Nived et al.
Lupus 2007;16:701-706.
ABSTRACT
Reduction in Estimated Risk for Coronary Artery Disease After Use of Ezetimibe with a Statin
Sampalis et al.
The Annals of Pharmacotherapy 2007;41:1345-1351.
ABSTRACT
| FULL TEXT
The Framingham Predictive Instrument in Chronic Kidney Disease
Weiner et al.
J Am Coll Cardiol 2007;50:217-224.
ABSTRACT
| FULL TEXT
Cross-Classification of Microalbuminuria and Reduced Glomerular Filtration Rate: Associations Between Cardiovascular Disease Risk Factors and Clinical Outcomes
Foster et al.
Arch Intern Med 2007;167:1386-1392.
ABSTRACT
| FULL TEXT
2007 Guidelines for the Management of Arterial Hypertension: The Task Force for the Management of Arterial Hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC)
Authors/Task Force Members: et al.
Eur Heart J 2007;0:ehm236v1-75.
FULL TEXT
Screening for cardiovascular risk factors in a dental setting
Greenberg et al.
Journal of the American Dental Association 2007;138:798-804.
ABSTRACT
| FULL TEXT
Guidelines on diabetes, pre-diabetes, and cardiovascular diseases: full text: The Task Force on Diabetes and Cardiovascular Diseases of the European Society of Cardiology (ESC) and of the European Association for the Study of Diabetes (EASD)
Authors/Task Force Members et al.
Eur Heart J Suppl 2007;9:C3-C74.
FULL TEXT
Osteopenia
Khosla and Melton
NEJM 2007;356:2293-2300.
FULL TEXT
Prediction of Incident Diabetes Mellitus in Middle-aged Adults: The Framingham Offspring Study
Wilson et al.
Arch Intern Med 2007;167:1068-1074.
ABSTRACT
| FULL TEXT
Long-Term Prognosis Associated With Coronary Calcification: Observations From a Registry of 25,253 Patients
Budoff et al.
J Am Coll Cardiol 2007;49:1860-1870.
ABSTRACT
| FULL TEXT
Risk Factors, Risk Prediction, and the Apolipoprotein B-Apolipoprotein A-I Ratio
Berkwits and Guallar
ANN INTERN MED 2007;146:677-679.
FULL TEXT
Framingham, SCORE, and DECODE Risk Equations Do Not Provide Reliable Cardiovascular Risk Estimates in Type 2 Diabetes
Coleman et al.
Diabetes Care 2007;30:1292-1293.
FULL TEXT
Association of CFH Y402H and LOC387715 A69S With Progression of Age-Related Macular Degeneration
Seddon et al.
JAMA 2007;297:1793-1800.
ABSTRACT
| FULL TEXT
Biomarkers for Prediction of Cardiovascular Events
Musunuru et al.
NEJM 2007;356:1472-1475.
FULL TEXT
The joint associations of occupational, commuting, and leisure-time physical activity, and the Framingham risk score on the 10-year risk of coronary heart disease
Hu et al.
Eur Heart J 2007;28:492-498.
ABSTRACT
| FULL TEXT
Cardiovascular risk prediction tools for populations in Asia
Asia Pacific Cohort Studies Collaboration
J. Epidemiol. Community Health 2007;61:115-121.
ABSTRACT
| FULL TEXT
ACCF/AHA 2007 Clinical Expert Consensus Document on Coronary Artery Calcium Scoring By Computed Tomography in Global Cardiovascular Risk Assessment and in Evaluation of Patients With Chest Pain: A Report of the American College of Cardiology Foundation Clinical Expert Consensus Task Force (ACCF/AHA Writing Committee to Update the 2000 Expert Consensus Document on Electron Beam Computed Tomography) Developed in Collaboration With the Society of Atherosclerosis Imaging and Prevention and the Society of Cardiovascular Computed Tomography
Greenland et al.
J Am Coll Cardiol 2007;49:378-402.
FULL TEXT
Effects of a Mediterranean-Style Diet on Cardiovascular Risk Factors
Martinez-Gonzalez et al.
ANN INTERN MED 2007;146:73-74.
FULL TEXT
Guidelines on diabetes, pre-diabetes, and cardiovascular diseases: executive summary: The Task Force on Diabetes and Cardiovascular Diseases of the European Society of Cardiology (ESC) and of the European Association for the Study of Diabetes (EASD)
Authors/Task Force Members et al.
Eur Heart J 2007;28:88-136.
FULL TEXT
Validity of an adaptation of the Framingham cardiovascular risk function: the VERIFICA study
Marrugat et al.
J. Epidemiol. Community Health 2007;61:40-47.
ABSTRACT
| FULL TEXT
Risk Factors for Mortality in Middle-aged Women
Tice et al.
Arch Intern Med 2006;166:2469-2477.
ABSTRACT
| FULL TEXT
Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review
Brindle et al.
Heart 2006;92:1752-1759.
ABSTRACT
| FULL TEXT
Estimation of 10-Year Risk of Fatal and Nonfatal Ischemic Cardiovascular Diseases in Chinese Adults
Wu et al.
Circulation 2006;114:2217-2225.
ABSTRACT
| FULL TEXT
Primary prevention of cardiovascular disease: a web-based risk score for seven British black and minority ethnic groups
Brindle et al.
Heart 2006;92:1595-1602.
ABSTRACT
| FULL TEXT
High Prevalence of Stroke Symptoms Among Persons Without a Diagnosis of Stroke or Transient Ischemic Attack in a General Population: The REasons for Geographic And Racial Differences in Stroke (REGARDS) Study.
Howard et al.
Arch Intern Med 2006;166:1952-1958.
ABSTRACT
| FULL TEXT
Cardiovascular disease risk factors in chronic kidney disease: overall burden and rates of treatment and control.
Parikh et al.
Arch Intern Med 2006;166:1884-1891.
ABSTRACT
| FULL TEXT
Has the Risk for Coronary Heart Disease Changed Among U.S. Adults?
Ajani and Ford
J Am Coll Cardiol 2006;48:1177-1182.
ABSTRACT
| FULL TEXT
Race and Ethnicity in Medical Research: Requirements Meet Reality
Winker
J Law Med Ethics 2006;34:520-525.
Efficacy and Safety in Clinical Trials in Cardiovascular Disease
Cohn
J Am Coll Cardiol 2006;48:430-433.
ABSTRACT
| FULL TEXT
Deficiencies of Cardiovascular Risk Prediction Models for Type 1 Diabetes
Zgibor et al.
Diabetes Care 2006;29:1860-1865.
ABSTRACT
| FULL TEXT
A 39-year-old woman with hypercholesterolemia.
Mittleman
JAMA 2006;296:319-326.
ABSTRACT
| FULL TEXT
Predicting cardiovascular risk: so what do we do now?
Lloyd-Jones and Tian
Arch Intern Med 2006;166:1342-1344.
FULL TEXT
An assessment of incremental coronary risk prediction using C-reactive protein and other novel risk markers: the atherosclerosis risk in communities study.
Folsom et al.
Arch Intern Med 2006;166:1368-1373.
ABSTRACT
| FULL TEXT
The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women
Cook et al.
ANN INTERN MED 2006;145:21-29.
ABSTRACT
| FULL TEXT
Prediction of Coronary Heart Disease in a Population With High Prevalence of Diabetes and Albuminuria: The Strong Heart Study
Lee et al.
Circulation 2006;113:2897-2905.
ABSTRACT
| FULL TEXT
Effectiveness and efficiency of different guidelines on statin treatment for preventing deaths from coronary heart disease: modelling study
Manuel et al.
BMJ 2006;332:1419.
ABSTRACT
| FULL TEXT
Derivation and Validation of a Prediction Score for Major Coronary Heart Disease Events in a U.K. Type 2 Diabetic Population.
Donnan et al.
Diabetes Care 2006;29:1231-1236.
ABSTRACT
| FULL TEXT
Prevention of cardiovascular diseases: focus on modifiable cardiovascular risk
El Fakiri et al.
Heart 2006;92:741-745.
ABSTRACT
| FULL TEXT
Reduction in Allostatic Load in Older Adults Is Associated With Lower All-Cause Mortality Risk: MacArthur Studies of Successful Aging
Karlamangla et al.
Psychosom. Med. 2006;68:500-507.
ABSTRACT
| FULL TEXT
|
