Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction
Background This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML). Methods and Results Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 20...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-11-01
|
| Series: | Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease |
| Subjects: | |
| Online Access: | https://www.ahajournals.org/doi/10.1161/JAHA.120.019697 |
| _version_ | 1811310958822293504 |
|---|---|
| author | Arman Kilic Robert H. Habib James K. Miller David M. Shahian Joseph A. Dearani Artur W. Dubrawski |
| author_facet | Arman Kilic Robert H. Habib James K. Miller David M. Shahian Joseph A. Dearani Artur W. Dubrawski |
| author_sort | Arman Kilic |
| collection | DOAJ |
| description | Background This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML). Methods and Results Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 2017 were included. ML models for operative mortality and major morbidity were previously developed using extreme gradient boosting. Concordance and discordance in predicted risk between ML and STS models were defined using equal‐size tertile‐based thresholds of risk. Calibration metrics and discriminatory capability were compared between concordant and discordant patients. A total of 243 142 patients were included. Nearly all calibration metrics were improved in cases of concordance. Similarly, concordance indices improved substantially in cases of concordance for all models with the exception of deep sternal wound infection. The greatest improvements in concordant versus discordant cases were in renal failure: ML model (concordance index, 0.660 [95% CI, 0.632–0.687] discordant versus 0.808 [95% CI, 0.794–0.822] concordant) and STS model (concordance index, 0.573 [95% CI, 0.549–0.576] discordant versus 0.797 [95% CI, 0.782–0.811] concordant) (each P<0.001). Excluding deep sternal wound infection, the concordance indices ranged from 0.549 to 0.660 for discordant cases and 0.674 to 0.808 for concordant cases. Conclusions Supplementing ML models with existing STS models for surgical aortic valve replacement may have an important role in risk prediction and should be explored further. In particular, for the roughly 25% to 50% of patients demonstrating discordance in estimated risk between ML and STS, there appears to be a substantial decline in predictive performance suggesting vulnerability of the existing models in these patient subsets. |
| first_indexed | 2024-04-13T10:09:31Z |
| format | Article |
| id | doaj.art-3861eabb8530449181e695aa601676a0 |
| institution | Directory Open Access Journal |
| issn | 2047-9980 |
| language | English |
| last_indexed | 2024-04-13T10:09:31Z |
| publishDate | 2021-11-01 |
| publisher | Wiley |
| record_format | Article |
| series | Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease |
| spelling | doaj.art-3861eabb8530449181e695aa601676a02022-12-22T02:50:58ZengWileyJournal of the American Heart Association: Cardiovascular and Cerebrovascular Disease2047-99802021-11-01102210.1161/JAHA.120.019697Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved PredictionArman Kilic0Robert H. Habib1James K. Miller2David M. Shahian3Joseph A. Dearani4Artur W. Dubrawski5Division of Cardiac Surgery University of Pittsburgh Medical Center Pittsburgh PAThe Society of Thoracic Surgeons Research Center Chicago ILThe Robotics Institute Carnegie Mellon University Pittsburgh PADepartment of Surgery Massachusetts General HospitalHarvard Medical School Boston MADepartment of Cardiovascular Surgery Mayo Clinic Rochester MNThe Robotics Institute Carnegie Mellon University Pittsburgh PABackground This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML). Methods and Results Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 2017 were included. ML models for operative mortality and major morbidity were previously developed using extreme gradient boosting. Concordance and discordance in predicted risk between ML and STS models were defined using equal‐size tertile‐based thresholds of risk. Calibration metrics and discriminatory capability were compared between concordant and discordant patients. A total of 243 142 patients were included. Nearly all calibration metrics were improved in cases of concordance. Similarly, concordance indices improved substantially in cases of concordance for all models with the exception of deep sternal wound infection. The greatest improvements in concordant versus discordant cases were in renal failure: ML model (concordance index, 0.660 [95% CI, 0.632–0.687] discordant versus 0.808 [95% CI, 0.794–0.822] concordant) and STS model (concordance index, 0.573 [95% CI, 0.549–0.576] discordant versus 0.797 [95% CI, 0.782–0.811] concordant) (each P<0.001). Excluding deep sternal wound infection, the concordance indices ranged from 0.549 to 0.660 for discordant cases and 0.674 to 0.808 for concordant cases. Conclusions Supplementing ML models with existing STS models for surgical aortic valve replacement may have an important role in risk prediction and should be explored further. In particular, for the roughly 25% to 50% of patients demonstrating discordance in estimated risk between ML and STS, there appears to be a substantial decline in predictive performance suggesting vulnerability of the existing models in these patient subsets.https://www.ahajournals.org/doi/10.1161/JAHA.120.019697aortic valve replacementcomplicationsmachine learningmortalityrisk prediction |
| spellingShingle | Arman Kilic Robert H. Habib James K. Miller David M. Shahian Joseph A. Dearani Artur W. Dubrawski Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease aortic valve replacement complications machine learning mortality risk prediction |
| title | Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction |
| title_full | Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction |
| title_fullStr | Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction |
| title_full_unstemmed | Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction |
| title_short | Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction |
| title_sort | supplementing existing societal risk models for surgical aortic valve replacement with machine learning for improved prediction |
| topic | aortic valve replacement complications machine learning mortality risk prediction |
| url | https://www.ahajournals.org/doi/10.1161/JAHA.120.019697 |
| work_keys_str_mv | AT armankilic supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction AT roberthhabib supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction AT jameskmiller supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction AT davidmshahian supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction AT josephadearani supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction AT arturwdubrawski supplementingexistingsocietalriskmodelsforsurgicalaorticvalvereplacementwithmachinelearningforimprovedprediction |