Authors: Saif Al-Shamsi
Categories: Research Article, Coronary heart disease, Framingham, Risk prediction, United Arab Emirates, Validation
Source: BMC Family Practice
Primary prevention guidelines recommend the use of the Framingham risk score (FRS) to estimate the 10-year coronary heart disease (CHD) risk in patients without diabetes for statin eligibility. However, the FRS model has never been validated in an Arab population. Therefore, this study aimed to examine the clinical performance of the FRS model for predicting 10-year CHD risk in adult United Arab Emirates (UAE) nationals without diabetes.
This 10-year retrospective cohort study included patients from the primary care clinics and outpatient specialty departments of a large tertiary care hospital in Al-Ain, UAE. They were aged 30–79 without a baseline history of cardiovascular disease and diabetes. The FRS for each subject was calculated. Follow-up data on hard CHD (hCHD) events (myocardial infarction or coronary death) for each participant were collected from the baseline visit in 2008 until December 31, 2019. The area under the time-dependent receiver operating characteristic (ROC) curve (AUROC) was used to assess the FRS model discrimination. Calibration was measured by using the Hosmer-Lemeshow χ^2^ test and the calibration curve. The optimal cutoff-point for hCHD risk prediction was determined by ROC curve analysis.
A total of 554 participants were included. The mean age was 48.0 ± 12.8 years and 45% were men. The mean predicted FRS of the study cohort was 5.2% and approximately 7% were classified as high-risk (≥ 20% threshold) by the FRS model. During a median follow-up of 10.2 years (interquartile range, 7.8–11.0 years), 26 hCHD events occurred. The FRS model displayed reasonably good discrimination (time-dependent AUROC 0.83) and calibration in predicting hCHD (Hosmer-Lemeshow χ^2^ statistic 11.2, P = 0.191). Applying the 20% high-risk threshold, the FRS model had a sensitivity of only 37% in identifying patients at high-risk for an hCHD event over 10 years. While a 7.5% optimal cutoff-point improved the sensitivity to 74%.
The FRS can be used in the prediction of coronary risk among UAE nationals without diabetes, however, the recommended hCHD risk threshold for statin eligibility may be too high. Lowering the cutoff-point to 7.5% could improve the identification of patients for preventive treatment.
Keywords: Coronary heart disease, Framingham, Risk prediction, United Arab Emirates, Validation
More than nine million deaths are attributed to coronary heart disease (CHD) globally according to a 2016 World Health Organization report [1]. In the Middle East, CHD is by far the most serious public health concern [2, 3]. Premature deaths associated with CHD have tripled over the past decade [4] and in a recent United Arab Emirates (UAE) study, the cumulative incidence of acute CHD events among men with one or more vascular risk factors was noted to be 8.9% over nine years [5]. While the crude incidence of acute CHD over ten years was 4.7% among European men in the general population [6]. This incidence is attributable to the high prevalence of risk factors for CHD in the UAE. When adjusted for age, the prevalence of smoking, diabetes, hypertension (HTN), and dyslipidemia is 11, 25, 29, and 51%, respectively [7]. Optimal, target-oriented primary prevention could help in reducing the incidence of CHD in the UAE population.
The morbidity and mortality associated with CHD and its related risk factors motivated research into the development of CHD risk assessment tools [8]. These tools, aid clinicians in the identification and risk stratification of individuals who would benefit from primary preventive strategies [9]. The Framingham risk score (FRS) [10], introduced in 2001, is recommended by the National Cholesterol Education Program-Adult Treatment Panel-III (NCEP-ATP-III) guidelines for the identification of high-risk individuals for lipid-lowering treatment [9]. The FRS model considers six traditional risk factors, age, sex, smoking, HTN, high-density lipoprotein-cholesterol (HDL-C), and total cholesterol (TC) to estimate a person’s absolute 10-year risk of incident CHD [8–10]. This simplified risk model was developed mainly among middle-aged Caucasian-Americans [8] and has been validated in different ethnic populations with varying results [10–12] although never in an Arab population.
Using unvalidated risk prediction tools could overestimate risk and result in administering inappropriate treatment, while underestimation could lead to undertreatment. Healthcare providers need to be confident in the accuracy of risk prediction models when applying them to the local population. Hence, this study sought to examine the clinical performance of the FRS model for predicting 10-year CHD risk in adult UAE nationals.
Participants for this external validation study were selected from the primary care clinics and outpatient specialty departments of Tawam Hospital, a large government healthcare center, in Al-Ain, UAE. Electronic medical records (EMR) of 1054 Emirati participants aged 30–79 without a documented history of CHD, stroke, or peripheral arterial disease at baseline were retrospectively evaluated between April 1, 2008, and December 31, 2008. According to the NCEP-ATP-III guidelines, the FRS is not intended for patients with diabetes, which is considered as CHD risk equivalent [13, 14], therefore, 496 patients with diabetes and 4 additional participants who were lost to follow-up or with missing baseline data were excluded from the final analysis (Fig. 1).
Fig. 1 Cohort selection flow chart. UAE, United Arab Emirates; CVD, cardiovascular disease
Follow-up data on hard (hCHD) events, defined by the Framingham Heart Study as acute myocardial infarction, or coronary death [10], were collected through a review of the patient EMR from the 2008 baseline visit until December 31, 2019. The outcomes were confirmed according to clinicians’ diagnosis and death certificates. This study was approved by the Research Ethics Committees of Tawam Hospital and the UAE University (CRD 239/13).
The baseline variables included age, sex, TC, HDL-C, systolic and diastolic blood pressure, the pharmacological treatment for HTN, and history of smoking (current or previous smokers). An automated oscillometric sphygmomanometer was used to measure blood pressure by qualified nurses.
Blood pressure, TC, and HDL-C in the Emirati validation cohort were categorized and compared along with age, sex, and smoking history to the Framingham study cohort [10]. HTN was classified based on the Seventh Joint National Committee on Hypertension normal blood pressure and prehypertension, systolic ≤139 mmHg or diastolic ≤89 mmHg; stage 1 HTN, systolic 140–159 mmHg or diastolic 90–99 mmHg; stage 2 HTN, systolic ≥160 or diastolic ≥100 mmHg. The higher category was chosen if the systolic and diastolic blood pressures were in different categories [15]. Cutoff-points of TC (< 5.17 mmol/L, 5.17–6.18 mmol/L, ≥ 6.20 mmol/L) and HDL-C (≤ 1.53 mmol/L, ≥ 1.55 mmol/L) were used to categorize values according to the NCEP-ATP-III guidelines [9]. Lipid levels were analyzed by standard methods using a Beckman Coulter UniCel® DxC800 Synchron Clinical System.
Differences in the baseline categorical variables between the Emirati validation and Framingham study cohorts were compared using Fisher’s exact test (two-sided). The 10-year predicted hCHD risk was calculated for each subject using the FRS model [10]. The predictor variables selected for the FRS model were age, sex, TC, HDL-C, systolic blood pressure, medications for HTN, and history of smoking. The calculated probabilities of risk were categorized into low (< 10%), intermediate (10–19.9%) and high (≥ 20%) risk [9] and stratified by observed hCHD events.
Discrimination and calibration were assessed to determine the predictive performance of the FRS model for 10-year hCHD risk in this validation cohort. Discrimination evaluates the model’s ability to differentiate individuals with and without the outcome event, based on the calculated predicted risk. The discriminatory power of the FRS model was evaluated by determining the area under the time-dependent receiver operating characteristic (ROC) curve (AUROC) at 10 years by using the Kaplan-Meier method [16]. Good discrimination was achieved if the time-dependent AUROC value was > 0.75. Calibration, which determines how closely the observed risk fit the predicted probabilities, was measured graphically and by using the Hosmer-Lemeshow χ^2^ test [17]. The calculated 10-year predicted hCHD risk was plotted against the actual incidence of hCHD events by deciles based on the predicted risk. A well-calibrated model would result in its calibration curve being closer to the 45-degree line, while a χ^2^ value < 20 indicates good calibration (P > 0.05).
The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were examined at 10 and 20% calculated FRS risk thresholds using the time-dependent ROC curve. The optimal high-risk cutoff-point for the study cohort was determined by applying the closest top left point approach to the ROC curve [18].
Statistical analyses were conducted using R software version 3.6.0 (The R Foundation for Statistical Computing, Vienna, Austria) and IBM SPSS software, version 26.0 (IBM Corporation, Armonk, NY, USA). A P value (two-tailed) less than 0.05 was considered statistically significant.
The baseline characteristics of the Emirati validation cohort and the original Framingham study cohort are displayed in Table 1. The mean age of the validation cohort was 48.0 ± 12.8 years and approximately 45% were men. Compared to the Framingham cohort, the prevalence of stage 2 HTN, smoking, diabetes, hypercholesteremia, and elevated HDL-C levels was lower in the Emirati subjects. The mean calculated 10-year FRS for incident hCHD of the Emirati validation cohort was 5.2%.
The distribution of the risk categories of the FRS model and the actual hCHD events is shown in Fig. 2. More than 80% (455/554) of the Emirati validation cohort was classified as low risk by the FRS model.
Fig. 2 Comparison of risk categories of the FRS model and hCHD events observed. FRS, Framingham risk score; hCHD, hard coronary heart disease
In this Emirati validation cohort of 554 participants, 26 hCHD events occurred in total, during a median follow-up of 10.2 years (interquartile range, 7.8–11.0 years). Of the 26 individuals who experienced an hCHD event within 10 years, 42, 23, and 35% were categorized as low, intermediate, and high-risk, respectively, by the FRS model.
The time-dependent AUROC for the FRS model at 10 years was 0.83, standard error (SE) 0.04, (Fig. 3), indicating a good discriminatory ability to differentiate hCHD events from non-events. The calibration curve in Fig. 4 demonstrated, overall, good agreement between the observed risk and predicted risk. The Hosmer-Lemeshow χ^2^ statistic was 11.2 (P = 0.191).
Fig. 3 Time-dependent ROC curve analysis at 10 years using the FRS model in UAE nationals. ROC, receiver operating characteristic; FRS, Framingham risk score; UAE, United Arab Emirates
Fig. 4 Calibration plots of observed and predicted 10-year CHD events in UAE nationals using the FRS. CHD, coronary heart disease; UAE, United Arab Emirates; FRS, Framingham risk score
Table 2 depicts the time-dependent sensitivity, specificity, PPV, and NPV of the NCEP-ATP-III recommended high-risk threshold of 20% for hCHD 10-year risk and the 7.5% optimal cutoff-point determined by ROC curve analysis. Sensitivity at the 20% risk threshold and 7.5% optimal cutoff-point was 37 and 74%, respectively, while specificity was 94 and 77%, respectively.
This validation study examined the clinical performance of the FRS model in an external Emirati cohort. This study found the FRS model to have both reasonably good discrimination and calibration in predicting hCHD among UAE nationals without diabetes. However, the sensitivity of the recommended decision threshold of 20% for identifying high-risk individuals was relatively poor.
Since the NCEP-ATP-III guideline for cholesterol management was released in 2001, several studies have researched the generalizability of the FRS model for predicting 10-year hCHD risk in different ethnic populations. In a study that assessed the applicability of the FRS equation to non-Caucasian Americans, found that although the model discriminated well in Native Americans, Hispanic, and Japanese Americans, it overestimated their risk, particularly in higher predicted risk categories [10]. Similarly, in studies of Asian populations, the FRS was observed to have moderate discrimination and poor calibration, with greater variations noted in the higher deciles of risk [11, 19].
So far, there is no published literature on the performance of prediction models in an Arab population. Although this study provides the first evidence that the FRS (NCEP-ATP-III) model accurately predicts coronary risk in UAE nationals without diabetes, the recommended 20% high-risk threshold may be excessive. With this threshold, the FRS model correctly identified only approximately 37% of patients who subsequently experienced an hCHD event within 10 years as being at high-risk (sensitivity). A study conducted in India that included 740 CHD patients without diabetes, found that the FRS identified only 15.1% of these patients as high-risk [20]. This disproportionate underestimation of risk by the FRS model, when using the recommended risk thresholds, may lead to patients who need it most being deprived of the necessary preventive treatment. However, according to the current study’s ROC curve analysis, selecting the optimal cutoff-point of 7.5% could potentially correctly identify about 74% of UAE nationals who would experience an hCHD event within 10 years.
The results of this study may have clinical implications for CHD prevention in UAE nationals without diabetes. Cardiovascular risk prediction tools, such as the FRS model, have been widely recommended by international and local guidelines to better target primary preventive treatments, particularly lipid-lowering therapy, in susceptible individuals [15, 21]. However, the FRS model does not include emerging risk factors, such as renal failure, which is associated with increased cardiovascular risk among UAE nationals [5]. Therefore, developing novel risk prediction models based on local data would be more appropriate in populations with varying disease patterns and risk factors.
This study’s 10-year follow-up period and the criteria used to define predictor variables and hCHD events align with the original Framingham study. However, several limitations need to be considered. First, this study used ambulatory patients’ data derived from the EMR of a single large tertiary care center, therefore, the generalizability of the results to the broader UAE population may be limited. However, the overall mean 10-year cardiac risk of 5.2% estimated at baseline is comparable to the results of several large population-based studies in the UAE [21, 22]. Second, the original derivation cohort from the Framingham study did not include individuals aged < 30 years, therefore the FRS tool may be imprecise in this population group. Finally, selection bias may have occurred, on account of the study’s retrospective cohort design.
In summary, this study demonstrated the extendibility of the FRS (NCEP-ATP-III) model in the prediction of coronary risk in UAE nationals without diabetes. However, the recommended hCHD risk threshold of 20% for lipid-lowering initiation may be too high and could lead to undertreatment. In the absence of a locally developed cardiac risk prediction tool, the FRS model could be used by primary care providers. Lowering the hCHD high-risk threshold to 7.5% could improve the risk-stratification of patients for preventive treatment in the UAE.
Not applicable.
SAS contributed to the conceptualization and design of the study, data collection, statistical analysis, interpretation of the data, and writing of the manuscript. The author has read and approved the final version of the manuscript.
This work was supported by the College of Medicine and Health Sciences, United Arab Emirates University (No. 31 M325). The funders had no role in study design, data collection, and analysis, decision to publish, or preparation of the manuscript.
The data that support the findings of this study are available from the corresponding author upon reasonable request.
This study was approved by the Research Ethics Committees of Tawam Hospital and the United Arab Emirates University (CRD 239/13). The requirement for informed consent was waived because patient records and information were anonymized and de-identified before analysis.
Not applicable.
The author declares no conflicts of interest with respect to the authorship and/or publication of this article.
The data that support the findings of this study are available from the corresponding author upon reasonable request.