Authors: John DePaolo (1Dept of Surgery), Gina Biagetti (2Division of Vascular Surgery and Endovascular Therapy, Dept of Surgery), Renae Judy (1Dept of Surgery), Grace J. Wang (2Division of Vascular Surgery and Endovascular Therapy, Dept of Surgery), John Kelly (3Division of Cardiovascular Surgery, Dept of Surgery), Amit Iyengar (3Division of Cardiovascular Surgery, Dept of Surgery), Nicholas J. Goel (3Division of Cardiovascular Surgery, Dept of Surgery), Nimesh Desai (3Division of Cardiovascular Surgery, Dept of Surgery), Wilson Y. Szeto (3Division of Cardiovascular Surgery, Dept of Surgery), Joseph E. Bavaria (3Division of Cardiovascular Surgery, Dept of Surgery), Michael G. Levin (4Dept of Medicine, Division of Cardiology; 5Corporal Michael Crescenz VA Medical Center, Philadelphia, PA), Scott M. Damrauer (2Division of Vascular Surgery and Endovascular Therapy, Dept of Surgery; 5Corporal Michael Crescenz VA Medical Center, Philadelphia, PA; 6Dept of Genetics, Perelman School of Medicine, University of Pennsylvania; 7Penn Cardiovascular Institute, Perelman School of Medicine, University of Pennsylvania)
Categories: Article, Ascending Thoracic Aorta, Aortic Dilation, Genetic Risk Score, Statistical Models, Bayesian Prediction
Source: Circulation. Genomic and precision medicine
Authors: John DePaolo, Gina Biagetti, Renae Judy, Grace J. Wang, John Kelly, Amit Iyengar, Nicholas J. Goel, Nimesh Desai, Wilson Y. Szeto, Joseph E. Bavaria, Michael G. Levin, Scott M. Damrauer
Ascending thoracic aortic dilation is a complex heritable trait that involves modifiable and non-modifiable risk factors. Polygenic scores (PGS) are increasingly used to assess risk for complex diseases. The degree to which a PGS can improve aortic diameter prediction in diverse populations is unknown. Presently, we tested whether adding a PGS to clinical prediction algorithms improves performance in a diverse biobank.
The analytic cohort comprised 6,235 Penn Medicine Biobank participants with available echocardiography and clinical data linked to genome-wide genotype data. Linear regression models were used to integrate PGS weights derived from a genome wide association study of thoracic aortic diameter performed in the UK Biobank and were compared to the performance of the previously published AORTA Score.
Cohort participants had a median age of 61 years (IQR 53–70) and a mean ascending aortic diameter of 3.36 cm (SD 0.49). Fifty-five percent were male and 33% were genetically similar to African reference populations (AFR). Compared to the AORTA Score, which explained 30.6% (95%CI 29.9% to 31.4%) of the variance in aortic diameter, AORTA Score + UKB-derived PGS explained 33.1%, (95%CI 32.3% to 33.8%), the reweighted AORTA score explained 32.5% (95%CI 31.8% to 33.2%), and the reweighted AORTA Score + UKB-derived PGS explained 34.9% (95%CI 34.2% to 35.6%). When stratified by population, models including the UKB-derived PGS consistently improved upon the clinical AORTA Score among individuals genetically similar to European reference populations (EUR), but conferred minimal improvement among individuals genetically similar to AFR reference populations. Comparable performance disparities were observed in models developed to discriminate cases/non-cases of thoracic aortic dilation (≥4.0cm).
We demonstrated that inclusion of a UKB-derived PGS to the AORTA Score confers a clinically meaningful improvement in model performance only among individuals genetically similar to EUR reference populations and may exacerbate existing healthcare disparities.
Thoracic aortic dilation can lead to the development of thoracic aortic aneurysm (TAA), a life-threatening condition. Without treatment, TAA can progress to acute rupture or aortic dissection, which represent approximately 8% of all out-of-hospital cardiopulmonary arrests.^1^ For patients admitted to a hospital for emergent treatment of thoracic aortic aneurysm and dissection (TAAD), observed 30-day mortality is 39–49%.^2–5^ Despite the high mortality risk, most cases of ascending thoracic aortic dilation and subsequent aneurysm are asymptomatic and identified incidentally.^6^ When recognized, asymptomatic ascending thoracic aortic aneurysms can be monitored with serial imaging and treated with aggressive blood pressure control and elective surgery, yielding significantly improved outcomes when compared to emergency intervention.^7^
Genome-wide association studies (GWAS) have identified numerous genomic loci associated with both ascending aortic diameter and TAAD, and both traits are highly heritable.^8–10^ Across a range of cardiovascular diseases, polygenic scores (PGS) have become increasingly attractive as predictive tools to utilize in clinical settings to estimate individual level genetic risk for specific diagnoses.^11–13^ We and others have formulated PGSs to assess risk of increased ascending thoracic aortic diameter (AscAoD),^8,9^ and TAAD.^10^ Because these traits are heritable, a PGS may represent a clinically useful tool to identify individuals who warrant screening with non-invasive imaging. However, recent research has identified significant shortcomings in the application of PGS derived from mostly European populations to individuals outside of that population.^14–17^ This has raised important questions about whether particular PGS can be applied in a diverse clinical setting and if this application may perpetuate healthcare disparities between white and non-white individuals.
Recently, a clinical risk score to identify patients at elevated risk for ascending thoracic aortic dilation, referred to as the AORTA (aorta optimized regression for thoracic aneurysm) Score, was developed using cardiac magnetic resonance imaging (cMRI) measurements of AscAoD among individuals within the UK Biobank (UKB), and validated among subsets of individuals within the Framingham Heart Study (using computed tomography [CT] scans) and Mass General Brigham Biobank (using transthoracic echocardiography [TTE]).^18^ A subsequent study, referred to as AORTA Gene, integrated a PGS for AscAoD into the AORTA Score model and applied it to a largely homogenous population similar to the original AORTA Score study demonstrating significant model performance improvement with the inclusion of a PGS.^19^ In the present study, we aimed to test the validity of the AORTA score within a separate diverse population and importantly identify the limits of applying a PGS derived in a European population to a more diverse population using multiple methods to assess model performance.
The full methods are available as Supplemental Material. All participants enrolled in the PMBB and included in the present study provided informed consent. This study received ethical approval from the University of Pennsylvania Institutional Review Board under protocol# 813913. The data that support the findings of this study are available from the corresponding author upon reasonable request.
A total of 6,235 PMBB adult participants were included based on available data, and their clinical characteristics were compared to the cohort of 30,018 UKB participants from which the standard AORTA Score was derived (Table 1). The cohort comprised 2,064 (33%) individuals genetically similar to the African reference population (AFR) and 4171 (67%) individuals genetically similar to the European reference population (EUR). Fifty-five percent of the individuals were male, and the median age was 61 years (IQR 53 – 70). AscAoD ranged from 1.70 cm to 6.00 cm (Figure S1), and the mean was 3.36 cm. There were 728 individuals (11.7%) with a diameter ≥ 4 cm. Seventy-four percent of individuals had a history of hypertension, 63% had a history of hyperlipidemia, and 33% had a history of diabetes. Compared to the UKB training cohort from which the AORTA Score was derived, the PMBB cohort had increased prevalence of co-morbidities and larger average AscAoD (mean 3.36 cm in PMBB compared to 3.18 cm in UKB).^18^
To evaluate our models, we considered several different performance measures, including assessments of model calibration and predictive performance (Figure 1). We then sought to quantify how the addition of UKB-derived PGS impacted performance above baseline AORTA score models, including only clinical variables, and how this may vary across individuals of different genetic ancestries. Next, we considered the diagnostic performance of models in identifying individuals with AscAoD ≥ 4 cm, an established clinical threshold for increased surveillance. Finally, we used decision curve analysis to compare the net benefit of implementing each model clinically.
We assessed calibration of our primary models using linear regression to compare predicted and observed aortic diameter. For a 1 cm estimate for AscAoD using the AORTA Score, the observed aortic diameter was 1.00 cm (95% confidence interval [CI] 0.96 cm to 1.04 cm, P<0.001); all other models were similarly calibrated with slopes of 1.0 (Table S1). Predicted values for individual models were plotted against measured and residual values (Figure S2). Bland-Altman plots for each of the models demonstrated more balanced 95% limits of agreement with a range of −0.83 to 0.83 and conservative estimation errors (Figure S3).
Next, we quantified the variance in measured aortic diameter explained by each model using R-squared (RSQ). The addition of the UKB-derived PGS improved the proportion of variance explained of the AORTA Score from 30.6% (95%CI 29.9% to 31.4%) to 33.1%, (95%CI 32.3% to 33.8%) [Figure 2], and the reweighted AORTA Score from 32.5% (95%CI 31.8% to 33.2%) to 34.9% (95%CI 34.2% to 35.6%) (Table S2). Using Bayesian ANOVA analyses to quantify cross-model performance differences between models with and without the UKB-derived PGS included,^20^ there was an 89% probability of a practical improvement when the UKB-derived PGS was added to the AORTA Score (mean RSQ difference 2.5%, 95% credible interval 1.8% to 3.1%), and there was an 85% probability of a practical improvement when the UKB-derived PGS was added to the reweighted AORTA Score (mean RSQ difference 2.4%, 95% credible interval 1.8% to 3.0%) [Figure S4, Table S3].^21^ Model accuracy as measured by root-mean-square error (RMSE) showed a similar improvement trend with the addition of the UKB-derived PGS (Tables S4), however in Bayesian analysis there was >95% practical equivalence when the UKB-derived PGS was added to either the AORTA Score or reweighted AORTA Score [Table S5, Figure S5].
To test for population specific model performance the cohort was stratified by genetic similarity to reference populations from the 1000 Genomes project (Table S6),^22^ based on principal component analysis (Figure S6). Results among the EUR population mirrored those of the overall cohort. The addition of the UKB-derived PGS improved the explanation of variance of the AORTA Score from 29.9% (95%CI 29.1% to 30.8%) to 33.7% (95%CI 32.7% to 34.6%) [Figure 2], and the reweighted AORTA Score from 31.7% (95%CI 31.0% to 32.5%) to 35.2% (95%CI 34.4% to 36.0%) [Table S2]. In Bayesian analysis, there was a >95% probability of a practical improvement when the UKB-derived PGS was added to both the AORTA Score (mean RSQ difference 3.7%, 95% credible interval 3.0% to 4.5%) and the reweighted AORTA Score (mean RSQ difference 3.5%, 95% credible interval 2.8% to 4.2%) [Figure S7, Table S7]. However, there was >95% probability of practical equivalence in RMSE between models with and without the addition of the UKB-derived PGS (Figure S8, Table S8).
In the AFR population, the addition of the UKB-derived PGS provided no significant improvement in model performance. The AORTA Score explained 28.3% (95%CI 27.0% to 29.6%) of model variance, which was not significantly different compared to the AORTA Score + UKB-derived PGS (28.8% [95%CI 27.6% to 30.1%]) [Figure 2], reweighted AORTA Score (28.6% [95%CI 27.3% to 30.0%]), and reweighted AORTA Score + UKB-derived PGS (29.3% [95%CI 28.1% to 30.6%]) [Table S2]. In Bayesian analysis, there was a >95% probability of practical equivalence when the UKB-derived PGS was added to both the AORTA Score (mean RSQ difference 0.6%, 95% credible interval −0.5% to 1.6%) and the reweighted AORTA Score (mean RSQ difference 0.7%, 95% credible interval −0.4% to 2.7%) [Figure S9, Table S9]. In Bayesian analysis of RMSE, there was similar evidence of practical equivalence between models with and without the UKB-derived PGS (Figure S10, Table S10). In aggregation, these stratified results suggest that the UKB-derived PGS enhances model performance among EUR individuals but not among AFR individuals.
To determine if a trans-ancestry PGS derived from a GWAS of a similar trait might improve model explanation of variance in the AFR population, we derived a PGS from a TAAD GWAS performed in the more diverse Million Veterans Program (MVP).^10^ Integration of the MVP-derived TAAD PGS in the AORTA Score enhanced explanation of variance to an attenuated but similar degree (RSQ=31.5%, 95%CI 30.4% to 32.6%) as the addition of the UKB-derived AscAoD PGS (Table S11). However, the addition of the MVP-derived TAAD PGS to the AORTA Score provided no improved explanation of variance (RSQ=27.9%, 95%CI 26.5% to 29.3%) in the AFR population (Table S11), demonstrating that a PGS of a related trait derived from more diverse GWAS cohort does not enhance AORTA Score performance across diverse populations.
Finally, we assessed for differences in model performance by sex (Table S12). Among females only, the addition of the UKB-derived PGS improved the explanation of variance of the AORTA Score from 16.3%% (95%CI 15.2% to 17.3%) to 18.8% (95%CI 17.6% to 20.0%), and the reweighted AORTA Score from 17.7% (95%CI 16.6% to 18.8%) to 20.0% (95%CI 18.8% to 21.2%) [Table S13]. In Bayesian analysis, there was an 84% probability of a practical improvement when the UKB-derived PGS was added to the AORTA Score (mean RSQ difference 2.5%, 95% credible interval 1.6% to 3.5%) and a 70% probability of a practical improvement when the UKB-derived PGS was added to the reweighted AORTA Score (mean RSQ difference 2.3%, 95% credible interval 1.4% to 3.2%) [Figure S11, Table S14]. When the female cohort was stratified by genetic similarity to reference populations, a >95% probability of a practical improvement was observed among EUR females when the UKB-derived PGS was added to both the AORTA Score and reweighted AORTA Score (Figure S12, Table S15). However, among AFR females there was >95% probability of practical equivalence when the UKB-derived PGS was added to the to both the AORTA Score and reweighted AORTA Score (Figure S13, Table S16).
Among males only, the addition of the UKB-derived PGS improved the explanation of variance of the AORTA Score from 17.5% (95%CI 16.4% to 18.6%) to 20.8% (95%CI 19.6% to 22.0%), and the reweighted AORTA Score from 18.9% (95%CI 17.9% to 19.9%) to 22.2% (95%CI 21.1% to 23.3%) [Table S17]. In Bayesian analysis, a >95% probability of practical improvement when the UKB-derived PGS was added to both the AORTA Score (mean RSQ difference 3.3%, 95% credible interval 2.4% to 4.2%) and the reweighted AORTA Score (mean RSQ difference 3.3%, 95% credible interval 2.4% to 4.2%) [Figure S14, Table S18]. When the male cohort was stratified into genetically similar populations, a >95% probability of practical improvement was observed among EUR males when the UKB-derived PGS was added to both the AORTA Score and reweighted AORTA Score (Figure S15, Table S19), but among AFR males >95% practical equivalence was observed when the UKB-derived PGS was added to both the AORTA Score and reweighted AORTA Score (Figure S16, Table S20). Taken together, these findings suggest that the UKB-derived PGS improves model performance among EUR individuals independent of sex, but the addition of the UKB-derived PGS does not improve model performance among AFR males or females.
To assess the clinical performance of each model in predicting individuals with aortic diameters ≥ 4 cm, a clinically meaningful threshold at which serial imaging would be recommended, we performed logistic regression analysis using the covariates from the AORTA score and their interactions, including the UKB-derived AscAoD PGS where indicated, and analyzed receiver operator characteristic curves (ROC) and area under the ROC (AUROC). Similar to what was observed among all individuals (Figure 3A–3B), among the EUR populations the AORTA Score + UKB-derived PGS (AUROC = 0.782, 95%CI 0.775 to 0.790) outperformed the standard AORTA Score (AUROC = 0.761, 95%CI 0.754 to 0.768) [Figure 3C–3D, Table S21], and the reweighted AORTA Score + UKB-derived PGS (AUROC = 0.783, 95%CI 0.775 to 0.791) outperformed the reweighted AORTA Score (AUROC = 0.761 95%CI 0.754 to 0.768) [Figure S17C-S17D, Table S21]. In Bayesian analysis, there was an 81% probability of a practical improvement in AUROC between the AORTA Score and the AORTA score + UKB-derived PGS, and a 91% probability of a practical improvement in AUROC between the reweighted AORTA Score and the reweighted AORTA Score + UKB-derived PGS (Figure S18, Table S22).
Among the AFR population, the AORTA Score (AUROC = 0.816, 95%CI 0.802 to 0.831), AORTA Score + UKB-derived PGS (AUROC = 0.819, 95%CI 0.803 to 0.834) [Figure 3E–3F, Table S23], reweighted AORTA Score (AUROC = 0.800 95%CI 0.785 to 0.815), and reweighted AORTA Score + UKB-derived PGS (AUROC = 0.805, 95%CI 0.791 to 0.819) [Figure S17E-S17F, Table S23] each performed similarly with only a subtle improvement with the addition of a UKB-derived PGS. In Bayesian analysis, there was a >95% probability of AUROC practical improvement when comparing the AORTA Score to the AORTA Score + UKB-derived PGS, and the reweighted AORTA Score to the reweighted AORTA Score + UKB-derived PGS (Figure S19, Table S24). Taken together, these results suggest that the addition of the UKB-derived PGS to the standard and reweighted AORTA Scores significantly improves the ability of either model to predict AscAoD ≥ 4 cm among the EUR population but only marginally among the AFR population.
To investigate whether the UKB-derived PGS integration provides net clinical benefit to the AORTA Score, we performed decision curve analyses (DCA) using each logistic regression model. DCA allows the assessment of net clinical benefit which equates to the number of true positives screened less the number of false positives screened at a specific threshold probability, and compared to “screen all” or “screen none” models.^23^ In this evaluation, the screening test is a transthoracic echocardiogram, a low-risk, non-invasive, and inexpensive procedure.
In the EUR population, the addition of the UKB-derived PGS consistently contributed to clinical performance enhancement at a range of threshold probabilities between 0% and 25% (Figure 4A and S20A, Table S25). At the threshold probability of 15% for example, the net benefit of the AORTA Score + UKB-derived PGS (5.7 cases per 100 individuals screened) was higher than the standard AORTA Score (5.3 cases per 100 individuals screened), and the net benefit of the reweighted AORTA Score + UKB-derived PGS (5.8 cases per 100 individuals screened) was higher than the reweighted AORTA Score (5.4 cases per 100 individuals screened). These data suggest that the addition of the UKB-derived PGS enhance the overall clinical utility of the standard AORTA Score and reweighted AORTA Score models among EUR individuals.
Among AFR individuals, the addition of the UKB-derived PGS yielded limited and inconsistent benefit compared to either the standard or reweighted AORTA Score models (Figure 4B and S20B, Table S26). For example, at the same threshold probability of 15%, the net benefit of the AORTA Score + UKB-derived PGS (0.83 cases per 100 individuals screened) was lower than the standard AORTA Score (1.2 cases per 100 individuals screened), and the net benefit of the reweighted AORTA Score + UKB-derived PGS (1.0 cases per 100 individuals screened) was lower than the reweighted AORTA Score (1.3 cases per 100 individuals screened). These data contribute further evidence that the addition of the UKB-derived PGS to the AORTA Score model utilized to predict ascending thoracic aortic dilation in the AFR population does not add clinical benefit.
Using a published predictive model of ascending thoracic aortic dilation, we demonstrated that the addition of a UKB-derived PGS based on the largest available GWAS of AscAoD significantly improved standard AORTA Score and reweighted AORTA Score performance among individuals genetically similar to European reference populations. However, we found that the addition of the UKB-derived PGS offered minimal model improvement among individuals genetically similar to African reference populations. Notably, reweighting the AORTA Score based on local clinical characteristics improved model performance among EUR individuals but this improvement was also attenuated among AFR individuals.
Currently, there are no meaningful recommendations for population-based screening for dilated ascending thoracic aortas among individuals who lack a history of familial disease. Our replication of the standard AORTA Score in a diverse population demonstrates its effectiveness in predicting individuals who would benefit from screening for thoracic aortic dilation in a diverse biobank. DCA revealed the superiority of AORTA Score implementation over a “screen all” or the current “screen none” approach in both EUR and AFR populations. Prospective investigation is warranted to determine its clinical benefit.
The addition of the AscAoD UKB-derived PGS improved model performance across a range of analyses among the EUR study population including explanation of variance, discrimination of individuals with diameter ≥ 4 cm, and DCA. As AscAoD was found to be highly heritable in the primary GWAS (63%, 95%CI 60–67),^9^ these results are encouraging with respect to the potential benefit genetic data may have on clinical risk modeling. However, consistent with previous studies,^14–17^ we found that UKB-derived PGS performance is not portable across genetically diverse populations. In the AFR study population, addition of the UKB-derived PGS conferred minimal model improvement. This disparate effect stems from the GWAS of AscAoD being performed in a population almost entirely genetically similar to the European reference population. For genetic data to have an equitable effect in improving the AORTA Score, efforts should be taken to increase the diversity of individuals included in GWAS studies of ascending thoracic aortic diameter.
The stratified findings we report are important when considering the potential application of a predictive model in a diverse population. In their work, Pirruccello et al. demonstrate that the AORTA Gene, a version of the AORTA Score that incorporates a UKB-derived PGS, performs better than the standard AORTA Score in four separate cohorts, including UKB, Mass General Brigham Biobank (MGB), Framingham Heart Study (FHS), and All of Us biobank.^19^ In the cohorts where ascending thoracic aortic diameter was measured by TTE, model improvement was similar to our the explanation of variance improved in MGB (32.5% to 36.5%) and All of Us (29.9% to 34.9%) to a similar magnitude when compared to EUR PMBB participants (29.9% to 33.7%). However, ancestry/race was only considered within their MGB cohort analysis, and only 340 individuals who self-identified as Black were separately analyzed in a stratified manner. In this stratified analysis, they report no difference between AORTA Gene and AORTA Score RSQ, but they do report a nominally significant difference in AUROC between the two models. Our stratified analysis of AFR individuals represents a more than six-fold population increase (2,064 compared to 340) when compared to the MGB analysis. Given the >95% probability of practical equivalence between the scores with and without the UKB-derived PGS components in the PMBB AFR population, our results suggest that the UKB-derived PGS has little value among individuals genetically similar to the 1000 Genomes AFR superpopulation. This leads us to conclude that extrapolation of AORTA Gene results from a homogenous biobank may not portend model improvement among diverse populations given the lack a model improvement in our study.
Our observations lead us to conclude that the standard AORTA Score, which includes only clinical variables, represents the most unbiased screening method for ascending aortic dilation in a diverse population and clinical implementation. As GWAS data expands to include more diverse populations, integration of a PGS into the AORTA Score should be revisited. Once the addition of a PGS has more diverse benefit, the AORTA Score + PGS may be employed at healthcare centers with large biobanks in prospective studies similar to Recall by Genotype (RbG) initiatives where all enrolled individuals are screened using a model that includes a PGS and those at elevated risk are contacted and offered echocardiographic screening.
This study has several limitations. First, the PMBB is enriched for individuals with cardiovascular disease, including individuals diagnosed with TAAs due to the robustness of the aortic surgery practice at Penn, which may impact the application of our findings to other cohorts. Second, while all TTEs were read and reported by cardiologists in the Penn clinical echocardiography lab, which is accredited by the Intersocietal Commission for the Accreditation of Echocardiography Labs and staffed by cardiologists who are board-certified in echocardiography, the TTEs were not interpreted by the same cardiologists and small differences in measurement and interpretation may exist between studies. Third, both the AORTA Score and the PGS for ascending thoracic aortic dilation were derived from the UKB whose participants are largely healthier and less diverse than many of the major urban centers within the US from which regional biobanks like the PMBB are constructed. While reweighting the AORTA Score to account for local differences in the prevalence of risk factors for aortic dilation led to improvements in model performance, the optimal strategy for developing a generalizable predictive model for aortic dilation remains unclear. Finally, this study was also limited to a single biobank. While we note disparities in model performance by genetic ancestry, evaluation of AORTA Score + UKB-derived PGS models in other cohorts is necessary to fully understand how these disparities may extend to other population subgroups. Further investigation among other cohorts will contribute to assessing the benefit of including the UKB-derived PGS as a covariate within the AORTA Score.
Our findings suggest that the standard and reweighted AORTA Score models both perform better among the EUR population when a UKB-derived PGS is incorporated as a covariate. This improvement is not observed in the AFR population. Inclusion of a UKB-derived PGS at this time would not be equitable in a diverse population and may exacerbate healthcare disparities already in existence. As GWAS data for AscAoD expands, implementation of an AORTA Score with genetic data should be revisited. At this time, implementation of the standard AORTA Score in prospective studies is warranted.