Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention

Chad Zack, C. Senecal, Y. Kinar, Yaakov Metzger, Yoav Bar-Sinai, R. Jay Widmer, Ryan Lennon, Mandeep Singh, Malcolm R. Bell, Amir Lerman, R. Gulati

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objectives: This study sought to determine whether machine learning can be used to better identify patients at risk for death or congestive heart failure (CHF) rehospitalization after percutaneous coronary intervention (PCI). Background: Contemporary risk models for event prediction after PCI have limited predictive ability. Machine learning has the potential to identify complex nonlinear patterns within datasets, improving the predictive power of models. Methods: We evaluated 11,709 distinct patients who underwent 14,349 PCIs between January 2004 and December 2013 in the Mayo Clinic PCI registry. Fifty-two demographic and clinical parameters known at the time of admission were used to predict in-hospital mortality and 358 additional variables available at discharge were examined to identify patients at risk for CHF readmission. For each event, we trained a random forest regression model (i.e., machine learning) to estimate the time-to-event. Eight-fold cross-validation was used to estimate model performance. We used the predicted time-to-event as a score, generated a receiver-operating characteristic curve, and calculated the area under the curve (AUC). Model performance was then compared with a logistic regression model using pairwise comparisons of AUCs and calculation of net reclassification indices. Results: The predictive algorithm identified a high-risk cohort representing 2% of all patients who had an in-hospital mortality of 45.5% (95% confidence interval: 43.5% to 47.5%) compared with a risk of 2.1% for the general population (AUC: 0.925; 95% confidence interval: 0.92 to 0.93). Advancing age, CHF, and shock on presentation were the leading predictors for the outcome. A high-risk group representing 1% of all patients was identified with 30-day CHF rehospitalization of 8.1% (95% confidence interval: 6.3% to 10.2%). Random forest regression outperformed logistic regression for predicting 30-day CHF readmission (AUC: 0.90 vs. 0.85; p = 0.003; net reclassification improvement: 5.14%) and 180-day cardiovascular death (AUC: 0.88 vs. 0.81; p = 0.02; net reclassification improvement: 0.02%). Conclusions: Random forest regression models (machine learning) were more predictive and discriminative than standard regression methods at identifying patients at risk for 180-day cardiovascular mortality and 30-day CHF rehospitalization, but not in-hospital mortality. Machine learning was effective at identifying subgroups at high risk for post-procedure mortality and readmission.

Original languageEnglish (US)
Pages (from-to)1304-1311
Number of pages8
JournalJACC: Cardiovascular Interventions
Volume12
Issue number14
DOIs
StatePublished - Jul 22 2019

Fingerprint

Percutaneous Coronary Intervention
Heart Failure
Area Under Curve
Hospital Mortality
Logistic Models
Confidence Intervals
Mortality
Machine Learning
ROC Curve
Registries
Shock
Demography
Population
Forests

All Science Journal Classification (ASJC) codes

  • Cardiology and Cardiovascular Medicine

Cite this

Zack, Chad ; Senecal, C. ; Kinar, Y. ; Metzger, Yaakov ; Bar-Sinai, Yoav ; Widmer, R. Jay ; Lennon, Ryan ; Singh, Mandeep ; Bell, Malcolm R. ; Lerman, Amir ; Gulati, R. / Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention. In: JACC: Cardiovascular Interventions. 2019 ; Vol. 12, No. 14. pp. 1304-1311.
@article{a04b2bb526164924971334c061182c48,
title = "Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention",
abstract = "Objectives: This study sought to determine whether machine learning can be used to better identify patients at risk for death or congestive heart failure (CHF) rehospitalization after percutaneous coronary intervention (PCI). Background: Contemporary risk models for event prediction after PCI have limited predictive ability. Machine learning has the potential to identify complex nonlinear patterns within datasets, improving the predictive power of models. Methods: We evaluated 11,709 distinct patients who underwent 14,349 PCIs between January 2004 and December 2013 in the Mayo Clinic PCI registry. Fifty-two demographic and clinical parameters known at the time of admission were used to predict in-hospital mortality and 358 additional variables available at discharge were examined to identify patients at risk for CHF readmission. For each event, we trained a random forest regression model (i.e., machine learning) to estimate the time-to-event. Eight-fold cross-validation was used to estimate model performance. We used the predicted time-to-event as a score, generated a receiver-operating characteristic curve, and calculated the area under the curve (AUC). Model performance was then compared with a logistic regression model using pairwise comparisons of AUCs and calculation of net reclassification indices. Results: The predictive algorithm identified a high-risk cohort representing 2{\%} of all patients who had an in-hospital mortality of 45.5{\%} (95{\%} confidence interval: 43.5{\%} to 47.5{\%}) compared with a risk of 2.1{\%} for the general population (AUC: 0.925; 95{\%} confidence interval: 0.92 to 0.93). Advancing age, CHF, and shock on presentation were the leading predictors for the outcome. A high-risk group representing 1{\%} of all patients was identified with 30-day CHF rehospitalization of 8.1{\%} (95{\%} confidence interval: 6.3{\%} to 10.2{\%}). Random forest regression outperformed logistic regression for predicting 30-day CHF readmission (AUC: 0.90 vs. 0.85; p = 0.003; net reclassification improvement: 5.14{\%}) and 180-day cardiovascular death (AUC: 0.88 vs. 0.81; p = 0.02; net reclassification improvement: 0.02{\%}). Conclusions: Random forest regression models (machine learning) were more predictive and discriminative than standard regression methods at identifying patients at risk for 180-day cardiovascular mortality and 30-day CHF rehospitalization, but not in-hospital mortality. Machine learning was effective at identifying subgroups at high risk for post-procedure mortality and readmission.",
author = "Chad Zack and C. Senecal and Y. Kinar and Yaakov Metzger and Yoav Bar-Sinai and Widmer, {R. Jay} and Ryan Lennon and Mandeep Singh and Bell, {Malcolm R.} and Amir Lerman and R. Gulati",
year = "2019",
month = "7",
day = "22",
doi = "10.1016/j.jcin.2019.02.035",
language = "English (US)",
volume = "12",
pages = "1304--1311",
journal = "JACC: Cardiovascular Interventions",
issn = "1936-8798",
publisher = "Elsevier Inc.",
number = "14",

}

Zack, C, Senecal, C, Kinar, Y, Metzger, Y, Bar-Sinai, Y, Widmer, RJ, Lennon, R, Singh, M, Bell, MR, Lerman, A & Gulati, R 2019, 'Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention', JACC: Cardiovascular Interventions, vol. 12, no. 14, pp. 1304-1311. https://doi.org/10.1016/j.jcin.2019.02.035

Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention. / Zack, Chad; Senecal, C.; Kinar, Y.; Metzger, Yaakov; Bar-Sinai, Yoav; Widmer, R. Jay; Lennon, Ryan; Singh, Mandeep; Bell, Malcolm R.; Lerman, Amir; Gulati, R.

In: JACC: Cardiovascular Interventions, Vol. 12, No. 14, 22.07.2019, p. 1304-1311.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Leveraging Machine Learning Techniques to Forecast Patient Prognosis After Percutaneous Coronary Intervention

AU - Zack, Chad

AU - Senecal, C.

AU - Kinar, Y.

AU - Metzger, Yaakov

AU - Bar-Sinai, Yoav

AU - Widmer, R. Jay

AU - Lennon, Ryan

AU - Singh, Mandeep

AU - Bell, Malcolm R.

AU - Lerman, Amir

AU - Gulati, R.

PY - 2019/7/22

Y1 - 2019/7/22

N2 - Objectives: This study sought to determine whether machine learning can be used to better identify patients at risk for death or congestive heart failure (CHF) rehospitalization after percutaneous coronary intervention (PCI). Background: Contemporary risk models for event prediction after PCI have limited predictive ability. Machine learning has the potential to identify complex nonlinear patterns within datasets, improving the predictive power of models. Methods: We evaluated 11,709 distinct patients who underwent 14,349 PCIs between January 2004 and December 2013 in the Mayo Clinic PCI registry. Fifty-two demographic and clinical parameters known at the time of admission were used to predict in-hospital mortality and 358 additional variables available at discharge were examined to identify patients at risk for CHF readmission. For each event, we trained a random forest regression model (i.e., machine learning) to estimate the time-to-event. Eight-fold cross-validation was used to estimate model performance. We used the predicted time-to-event as a score, generated a receiver-operating characteristic curve, and calculated the area under the curve (AUC). Model performance was then compared with a logistic regression model using pairwise comparisons of AUCs and calculation of net reclassification indices. Results: The predictive algorithm identified a high-risk cohort representing 2% of all patients who had an in-hospital mortality of 45.5% (95% confidence interval: 43.5% to 47.5%) compared with a risk of 2.1% for the general population (AUC: 0.925; 95% confidence interval: 0.92 to 0.93). Advancing age, CHF, and shock on presentation were the leading predictors for the outcome. A high-risk group representing 1% of all patients was identified with 30-day CHF rehospitalization of 8.1% (95% confidence interval: 6.3% to 10.2%). Random forest regression outperformed logistic regression for predicting 30-day CHF readmission (AUC: 0.90 vs. 0.85; p = 0.003; net reclassification improvement: 5.14%) and 180-day cardiovascular death (AUC: 0.88 vs. 0.81; p = 0.02; net reclassification improvement: 0.02%). Conclusions: Random forest regression models (machine learning) were more predictive and discriminative than standard regression methods at identifying patients at risk for 180-day cardiovascular mortality and 30-day CHF rehospitalization, but not in-hospital mortality. Machine learning was effective at identifying subgroups at high risk for post-procedure mortality and readmission.

AB - Objectives: This study sought to determine whether machine learning can be used to better identify patients at risk for death or congestive heart failure (CHF) rehospitalization after percutaneous coronary intervention (PCI). Background: Contemporary risk models for event prediction after PCI have limited predictive ability. Machine learning has the potential to identify complex nonlinear patterns within datasets, improving the predictive power of models. Methods: We evaluated 11,709 distinct patients who underwent 14,349 PCIs between January 2004 and December 2013 in the Mayo Clinic PCI registry. Fifty-two demographic and clinical parameters known at the time of admission were used to predict in-hospital mortality and 358 additional variables available at discharge were examined to identify patients at risk for CHF readmission. For each event, we trained a random forest regression model (i.e., machine learning) to estimate the time-to-event. Eight-fold cross-validation was used to estimate model performance. We used the predicted time-to-event as a score, generated a receiver-operating characteristic curve, and calculated the area under the curve (AUC). Model performance was then compared with a logistic regression model using pairwise comparisons of AUCs and calculation of net reclassification indices. Results: The predictive algorithm identified a high-risk cohort representing 2% of all patients who had an in-hospital mortality of 45.5% (95% confidence interval: 43.5% to 47.5%) compared with a risk of 2.1% for the general population (AUC: 0.925; 95% confidence interval: 0.92 to 0.93). Advancing age, CHF, and shock on presentation were the leading predictors for the outcome. A high-risk group representing 1% of all patients was identified with 30-day CHF rehospitalization of 8.1% (95% confidence interval: 6.3% to 10.2%). Random forest regression outperformed logistic regression for predicting 30-day CHF readmission (AUC: 0.90 vs. 0.85; p = 0.003; net reclassification improvement: 5.14%) and 180-day cardiovascular death (AUC: 0.88 vs. 0.81; p = 0.02; net reclassification improvement: 0.02%). Conclusions: Random forest regression models (machine learning) were more predictive and discriminative than standard regression methods at identifying patients at risk for 180-day cardiovascular mortality and 30-day CHF rehospitalization, but not in-hospital mortality. Machine learning was effective at identifying subgroups at high risk for post-procedure mortality and readmission.

UR - http://www.scopus.com/inward/record.url?scp=85068396486&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068396486&partnerID=8YFLogxK

U2 - 10.1016/j.jcin.2019.02.035

DO - 10.1016/j.jcin.2019.02.035

M3 - Article

C2 - 31255564

AN - SCOPUS:85068396486

VL - 12

SP - 1304

EP - 1311

JO - JACC: Cardiovascular Interventions

JF - JACC: Cardiovascular Interventions

SN - 1936-8798

IS - 14

ER -