Machine Learning and Clinical Predictors of Mortality in Cardiac Arrest Patients: A Comprehensive Analysis

Introduction

Cardiac arrest (CA), a critical public health issue, manifests differently when occurring inside versus outside hospital settings. Out-of-hospital cardiac arrest (OHCA) presents significant challenges globally, with an incidence among adults ranging from 40 to 52 per 100 000 person-years [1,2]. Despite emergency interventions, only about 22% of OHCA patients survive to be admitted to the hospital, and the survival rate to hospital discharge is as low as 9% for this group [3].

In contrast, in-hospital cardiac arrest (IHCA) shows comparatively higher recovery rates. Approximately one-third to two-thirds of IHCA patients achieve return of spontaneous circulation (ROSC) after cardiopulmonary resuscitation [4,5]. Furthermore, survival rates to hospital discharge have been improving, currently standing around 25% in the United States and up to 35% in European countries [6,7]. Survival variations after CA are influenced by numerous factors, including the structure of the emergency medical system, the promptness of basic life support (BLS) initiation by witnesses, the quality of advanced life support (ALS), and post-resuscitation care in intensive care units (ICU) [8]. To manage these critical situations, mortality prediction tools are frequently utilized in clinical practice for patients admitted to the ICU with life-threatening conditions [9].

In recent years, there has been rising interest in use of machine learning (ML) for medical purposes to develop a more personalized treatment approach based on more precise assessment of survival rates. The high applicability of ML algorithms stems from the fact that, as opposed to classic statistical methods, they simulate the process of learning and adapting (in some cases even through forgetting information) to the variability in the analyzed data. This important feature of ML divulges into a more optimal fit of such models to the empiric data. Contemporary medicine utilizes ML algorithms to assess the relationship between in-hospital survival and the various factors associated with the features of cardiac arrest and its comorbidities [10–15], treatment [16–19], demography, or even time- and date-related parameters [20]. Such multivariate analyses are performed to enable assessment of the odds of death in each patient; therefore, allowing ICUs to focus on the most endangered individuals [21,22]. The potential of ML in improving ICU outcomes extends beyond mere prediction – it enhances the decision-making process, tailoring interventions that are specific to the physiological profile of each patient. Recent studies illustrate how ML algorithms can optimize the treatment protocols by analyzing real-time data, thus potentially increasing survival rates in critical care settings. For instance, ML-based tools have shown promise in predicting severe complications and optimizing resource allocation in ICUs, which are crucial for improving outcomes in cardiac arrest cases [23,24]. Furthermore, ongoing advancements in ML are paving the way for more sophisticated models that integrate continuous data streams from ICU monitoring devices, offering deeper insights into patient status and likely trajectories [25].

Taking into account the potential of machine learning to significantly enhance outcomes in ICU settings, this study was designed to deepen our understanding of cardiac arrest mechanisms and their outcomes. Our research focused on investigating the differences in clinical parameters across asystole/pulseless electrical activity (PEA) and ventricular fibrillation/pulseless ventricular tachycardia (VF/pVT). We aimed to delineate how these parameters influence patient survival, define the critical conditions that contribute to mortality, and assess the odds of death, with a particular emphasis on identifying significant interactions between these conditions. Additionally, we explore the application of inherently explainable machine learning algorithms to predict survival status in patients after cardiac arrest, assessing the effectiveness of these tools in predicting outcomes based on the complex data available in ICU settings.

Material and Methods

STUDY DESIGN AND SETTING:

We performed a retrospective analysis of 161 medical records of patients admitted due to CA to the adult ICU at the University Hospital (Wrocław, Poland) between January 2017 and February 2022 (ICD10: I46). The study followed the STROBE guidelines (Strengthening the Reporting of Observational Studies in Epidemiology).

STUDY POPULATION AND DATA:

We analyzed all the patients who met the inclusion criteria (admitted to the ICU after CA, age ≥18 years old, and CA not caused by trauma or suicide attempt). The analysis included data such as age, sex, body mass index (BMI), Nutrition Risk Screening (NRS-2002), comorbidities, in-hospital mortality, and laboratory results including lymphocyte (LYM); procalcitonin (PCT); total cholesterol (TChol), triglycerides (TG), thyroid-stimulating hormone (TSH), high-sensitivity C-reactive protein (hsCRP), potassium (K), sodium (NA), and albumin. In-hospital mortality was defined as death from any cause occurring within the confines of the hospital, up until the time of discharge from the ICU.

We used as an auxiliary assessment of differences in the continuous variables. BMI was assessed as non-obese (BMI <30) and obese (BMI ≥30). Blood for laboratory tests was drawn by a nurse at the time of admission to the ICU. The physician admitting the patient to the ICU decided which tests the patient needed.

STATISTICAL ANALYSIS:

The analysis was performed based on our previous study in the same population sample, in which the association between categorical parameters (such as comorbidities) was studied [26].

Data pre-processing and visualization was performed with the use of Python 3.9.13. Statistical analyses were performed with use of Python 3.9.13 or STATISTICA 13.3 (package on license of Wrocław Medical University). The following Python packages were utilized: numpy 1.23.0, pandas 1.4.3, scipy 1.9.3, statsmodels 0.13.2, zepid 0.9.1, matplotlib 3.6.0, and dtreeviz 1.4.1. Logistic regression models were derived in the STATISTICA 13.3 package, while the machine learning model (Random Forest Classifier) was trained and tested with use of Python 3.9.13. (scikit-learn 1.1.3 package). Statistical inference was based on α=0.05.

Analysis of the distribution of values of variables, their scale of measurement, and the incidence of outliers or extreme values in the dataset was taken into account in the process of selecting the most suitable methods for statistical inference. Differences between values after grouping by different categories were checked with use of the Mann-Whitney U test.

The first algorithm used in survival status-related classification was logistic regression with effect coding. The initial set of variables was chosen based on the amount of missing data to minimalize loss of information. Information on this set of variables and the derivation of the base model (on which this study is based) is shown in Supplementary Materials (Supplementary Table 1) of our previous study on the same population sample [26]. The univariate analysis and interaction assessment formed the basis for implementing multivariate models. For the purpose of easier interpretability of the models, featured continuous variables, taking part in interactions within a model, were centered (value – mean value). The assumption of linearity between the predictors and the logit was checked using the Box-Tidwell test. Goodness of fit was assessed with: Hosmer-Lemeshow test, Akaike Information Criterion (AIC), Bayesian Information Criterion, and Nagelkerke’s pseudo-R2. Hypothesis that β=0 was tested with the Lagrange multiplier (score) test. Prediction power was analyzed based on the assessment of testing: AUC computed with 10-fold cross-validation. The best model was then described by metrics: including precision, recall, specificity, and accuracy, with use of the optimal cut-off value based on Youden’s J index.

Random Forest Classifier was the second algorithm used in the aforementioned classification. The method is based on creating a random set of decision trees (forming a forest). While originally the classifier is used to predict patient states (such as mortality) based on the whole ensemble of these decision trees, this study utilized the algorithm to choose a decision tree of the best fit from all the random trees in the ensemble. The algorithm was fit to the same dataset, and the same set of initial variables was used to assess which variables were featured the most (eg, ‘feature importance’) in all of the trees in the forest. Before fitting the algorithm to the data, the dataset was randomly shuffled and divided into training (65%) and testing (35%) subsets. Bootstrap samples were used when building trees. The classification was performed with use of 100 estimators (trees), among which the 2 best were selected for visualization (based on testing: accuracy or AUC, Supplementary Table 2). These 2 models were subsequently validated with use of 10-fold cross-validation performed on the whole dataset. The maximum tree depth was 5. The square root of the total number of used features was set as the maximum number of features to perform splits. Splits were performed if n ≥2. Leaves of the trees consisted of at least 1 record. The quality of splits was assessed with Gini impurity:

in which n is the count of predicted class categories and p_i is the proportion of the i^th class category. Feature importance was assessed based on the accumulation of impurity decrease within the trees.

Results

Characteristics of analyzed population sample

CHARACTERISTICS OF ANALYZED POPULATION SAMPLE:

The patients were aged 26–88 years old (mean value 64.07, SD 14.74, median value 67, 1^st to 3^rd interquartile range 56–74). The count of male patients was approximately 2.04-fold higher than the count of female patients.

DIFFERENCES IN VALUES OF SELECTED PARAMETERS BETWEEN THE 2 CARDIAC ARREST MECHANISMS: The 2 mechanisms of cardiac arrest differed significantly in age (p≈0.005) and concentrations of albumin (p≈0.042), PCT (p≈0.021), and hsCRP (p<0.001). Patients with the asystole/PEA mechanism were older and exhibited an 4.91% lower median value of albumin compared to the VF/pVT patients. The values of PCT and hsCRP in the asystole/PEA group were markedly higher (3.30-fold or 3.98-fold higher, respectively). The descriptive statistics of qualitative parameters are shown in Table 1.

Male patients were more likely to suffer from the VF/pVT mechanism (p≈0.002, Table 2), showing over 3-fold higher odds compared to their female counterparts. Over 2/3 of the total inter-hospital cases of cardiac arrest were of asystole/PEA mechanism, which is significantly more frequent compared to the OHCA individuals (p≈0.002, Table 2). Interestingly, these findings are not associated with each other since the male-to-female ratio did not differ in context of cardiac arrest location (p≈0.570, Figure 1). Moreover, the 2 mechanisms were associated with comparable comorbidity status (Table 2).

DIFFERENCES IN VALUES OF SELECTED QUANTITATIVE PARAMETERS IN CONTEXT OF SURVIVAL AND CARDIAC ARREST MECHANISM: Among the non-survivors (Figure 2A), asystole/PEA patients showed higher PCT concentration (p≈0.022) but lower TG concentration compared to the VF/pVT patients (p≈0.028). Interestingly, among the survivors (Figure 2B), VF/pVT patients showed higher BMI (p≈0.012) and albumin concentration (p≈0.003) but significantly lower hsCRP concentration (p≈0.003) compared to the asystole/PEA patients.

Although there were no survival status-associated differences in PCT concentration among the VF/pVT patients (p≈0.566), the asystole/PEA mechanism was associated with markedly higher PCT concentration observed in non-survivors when compared to the survivors (p≈0.018, Figure 2C). Among the VF/pVT patients, survivors showed higher albumin concentration (p<0.001, Figure 2D). This difference did not occur among the asystole/PEA patients (p≈0.524). Descriptive statistics are shown in Table 3 (total sample size in each group: Asystole/PEA: 29 survivors, 61 non-survivors; VF/pVT: 40 survivors, 31 non-survivors).

LOGISTIC REGRESSION ANALYSIS EXPLORING THE ODDS OF DEATH AMONG DIFFERENT CARDIAC ARREST MECHANISM TYPES: The base model utilized 2 significant effects selected by an iterative procedure: cardiac arrest mechanism (p≈0.038) and hsCRP (p≈0.038) (Supplementary Table 3: model 1). The iterative derivation of this model could be seen in our previous study, although the cut-off for leaving the variables in the model in that study was higher. As the prediction value of the model was questionable (testing AUC 0.645±0.0650), this model was expanded with additional effects and interactions. Adding the information on BMI and serum albumin concentration values, sex, and several interactions to the base model increased the prediction accuracy. The final model was created based on 2 intermedial models (Supplementary Table 1 (models 2 and 3) and Supplementary Table 3).

The best model (Table 4) utilized sex, cardiac arrest mechanism, BMI, albumin and hsCRP concentrations, and the 2nd-degree interactions of cardiac arrest mechanism with BMI, albumin, and hsCRP. This model proved to correctly classify approximately 76.2% of patients (testing AUC 0.762±0.0466). Its metrics at the optimal cut-off point (Supplementary Table 4) were 0.830 precision, 0.650 recall, 0.843 specificity, and 0.739 accuracy.

Men who suffered from asystole/PEA cardiac arrest showed approximately 5.68-fold higher (p≈0.007) odds of death compared to women with the same cardiac arrest mechanism. Interestingly, no sex-related differences in these odds were observed among patients with asystole/PEA cardiac arrest mechanism (p≈0.356).

The odds of death among men of 27.42 BMI and 2.97 g/dl serum albumin concentration (mean values from the population sample) were 8-fold lower (p≈0.001) among the VF/pVT stratum when compared to asystole/PEA. The differences in odds of death in the context of the cardiac arrest mechanism were insignificant among women (p≈0.620).

The association between serum albumin concentration and BMI values and the odds of death differed depending on cardiac arrest mechanism. Upon a 1-unit increase of BMI, the VF/pVT-to-asystole/PEA ratio of odds of death decreased approximately 1.26-fold (p≈0.016). This observation was associated with 1.18-fold increase (p≈0.032) in these odds with every 1-unit increase in BMI among patients with asystole/PEA mechanism. Interestingly, upon increase in serum albumin concentration by 1 g/dl, the odds of death decreased by 2.85-fold (p≈0.038) among patients suffering from the VF/pVT cardiac arrest type. Odds ratios, depending on the strata, are shown in Supplementary Table 1 and Figure 3.

RANDOM FOREST CLASSIFIER – WHICH FEATURES ARE THE MOST IMPORTANT IN SURVIVAL STATUS CLASSIFICATION PERFORMED IN PATIENTS WITH CARDIAC ARREST?: As previously shown, none of the categorical variables on their own could perfectly distinguish non-survivors from the whole population sample. Continuous variables also did not allow such classification (Figure 4). The ensemble of 100 random trees failed to achieve an AUC score comparable to the score of the aforementioned logistic regression model (AUC: 0.674 for random forest vs 0.762 for logistic regression). The ensemble suffered from poor precision in detecting the non-survivors (0.52), while the recall was relatively good (0.75). The confusion matrix of the ensemble is given in Supplementary Figure 1. Upon testing the classifier with 10-fold cross-validation, the ensemble yielded the following metrics: 0.656±0.210 precision, 0.700±0.222 recall, 0.606±0.156 specificity, 0.651±0.127 accuracy, and 0.715±0.132 AUC. The 3 features (variables) of most importance were PCT (0.183), age (0.133), and hsCRP (0.128). Interestingly, albumin was ranked 4th with feature importance of 0.108. BMI, sex, and cardiac arrest mechanism proved to be of less importance (Figure 5).

Precision, recall, accuracy, and AUC scores of each estimator (tree) are shown in Supplementary Table 2. Despite the fact that the whole ensemble of trees failed to classify survival status satisfyingly, 2 individual trees achieved good accuracy (tree ID 28, testing subset accuracy: 0.7179) or AUC score comparable to the value estimated for the featured logistic regression model (tree ID 25, testing subset AUC: 0.7582).

The tree with the best accuracy (tree ID 28, Figure 6) achieved an exceptional precision score (0.8333) at the cost of rather mediocre recall (0.6521). Based on this, hsCRP and age were the most useful features for performing splits. Underweight patients with a BMI ≤17.51 and a hsCRP >151.07 mg/l, would be considered as survivors. The error in classification performed on the learning subset occurred solely among patients of with K ≤7.44 mmol/l and hsCRP ≤55.25 mg/l. Of patients aged ≤62.50, the VF/pVT cardiac arrest mechanism occurred mostly among the survivors.

The tree with the best AUC score (tree ID 25) was more complex (Figure 7). Its precision and recall scores were 0.8235 and 0.6086, respectively. It utilized PCT instead of hsCRP and, interestingly, used nutrition-associated parameters (albumin, NRS2002, and BMI) for performing splits. Patients with PCT >0.17 ng/ml, albumin ≤2.85 g/dl, NRS ≤3.50, and IHCA cardiac arrest location were classified as non-survivors.

The classification capability of both trees was then assessed with 10-fold cross-validation. The tree which utilized nutritional status-associated parameters (Figure 7) proved to be markedly superior, scoring 0.745±0.178 precision, 0.754±0.091 recall, 0.708±0.148 specificity, 0.739±0.101 accuracy, and 0.754±0.137 AUC. The other tree (Figure 6) scored: 0.689±0.196 precision, 0.690±0.149 recall, 0.648±0.204 specificity, 0.677±0.129 accuracy, and 0.674±0.129 AUC.

Discussion

STUDY LIMITATIONS:

The study had some limitations. Firstly, in some cases the patient data were incomplete, as it was not always possible to obtain an outcome due to the critical situation and severe condition of the patients. This limitation could have affected the accuracy of our predictive models, as missing data points might lead to less reliable or biased predictions. Secondly, the number of patients was small, although it should be noted that the study included patients who were admitted to the ICU after OHCA/IHCA and only a small number of patients achieved ROSC. This small sample size might limit the generalizability of our results and affect the statistical power of our analyses, possibly obscuring smaller yet clinically significant effects. A larger sample size would have allowed us to analyze more complex (higher-degree) interactions between variables, as well as to investigate whether the interactions shown in this study are biased due to being modulated by other variables. Moreover, as the model studied only the categorical mortality status (survival or its lack), its use (after validation and further model development) would be limited in case the hospital would want to strategize the treatment based on the admission-to-death time. Although the mentioned time was investigated in our previous study, interactions were not included in that (multivariate) model [26]. The data in this study were analyzed retrospectively. Due to the anonymized data, information on long-term survival could not be obtained. Furthermore, specific details such as the exact cause and duration of the cardiac arrests were often not recorded, particularly for patients transferred from other institutions, which might have affected interpretation of associated risk factors and outcomes described in our findings. Also, variations in clinical practice over time and between different providers at the same institution could introduce additional variability into our findings.

Moreover, it should be emphasized that machine learning is only a tool which should be maintained by professionals, in accordance with the a priori clinical aims. In this study, random forest classifier, due to its subpar classification performance, was used to extract the optimal decision trees, based on accuracy and the AUC score. Both these metrics presume that sensitivity (% of true classifications among the non-survivors) and specificity (% of true classifications among the survivors) are equally important to the medical professionals who would be using these algorithms. In extreme cases in which sensitivity or specificity would have to be set close to 100% threshold (100% sensitivity if the algorithm would be used as a screening test, or 100% specificity if the algorithm was used as an auxiliary, after employing more sensitive assessment tools), the classification trees would not only be affected by different cut-off values, but would also probably utilize different variables. Therefore, assumptions need to be discussed among professionals who would want to further validate these, or similar, machine learning algorithms in real-life situations.

PRACTICAL IMPLICATIONS:

The preliminary findings from our study indicate significant potential for the use of machine learning algorithms in improving treatment outcomes for cardiac arrest cases. Despite the promising results, it is crucial to conduct further, expanded research on larger and more diverse patient groups. Such studies will allow for a more detailed analysis and validation of predictive models and their efficacy in real clinical settings. The development and refinement of these technologies, in close collaboration with medical professionals and data scientists, should focus on integration with existing healthcare systems and adaptation to specific needs and treatment protocols. This could significantly enhance the quality of healthcare and patient outcomes.