Stroke Prognostication in Patients Treated with Thrombolysis Using Random Forest

2.1. Data Collection

In this study, we leveraged patient data obtained at the initial presentation, comprising a comprehensive spectrum of clinical, laboratory, and non-contrast brain CT data sourced from the medical records and Picture Archiving Communication System (PACS) of Cipto Mangunkusumo Hospital, Jakarta. The inclusion criteria encompassed individuals who had encountered acute ischemic stroke and undergone thrombolysis treatment within a decade-long period spanning from November 2014 to February 2023, resulting in a total of 145 patients.

The clinical dataset employed in this research comprised six essential parameters: stroke onset, age, systolic and diastolic blood pressure, NIHSS scores upon admission (NIHSS_In), and 24 hours after therapy (NIHSS_24). With additional laboratory data of prothrombin time (PT), activated partial thromboplastin time (APTT), fibrinogen levels, and random blood glucose (RBG).

After acquiring non-contrast brain CT data with substantial slice thickness from PACS, an imperative preprocessing step was undertaken to optimize the CT dataset for subsequent analyses. The integration of clinical, laboratory, and post-processed CT data served as vital inputs for predicting the Δ NIHSS score, representing the difference between NIHSS_In and NIHSS_24 scores, crucial for understanding the progression and outcomes of acute ischemic stroke post-thrombolysis treatment.

The acquired Δ NIHSS scores are segmented into two distinctive classes—Significant reduction (≥4) and Insignificant reduction (<4)—outlined in Table 1. This classification schema was adopted to structure and categorize the responses observed post-thrombolysis treatment in other studies [13, 14]. In this study, a significant reduction was labelled as negative, meaning there is an improvement in clinical outcome 24 hours after thrombolysis therapy.

2.2. Data Preprocessing

The first step in processing the data was to use the nibabel library in Python to load the volumetric data in NIFTI format. This step was necessary due to the complexities of medical imaging datasets, particularly those from modalities like MRI or CT scans. Nibabel library ensures accurate handling of 3D medical imaging data, maintaining data integrity throughout the extraction process. (Fig. 1).

Subsequently, the 3D volumetric data was sampled into multiple 2D image slices. This transformation into 2D slices facilitated subsequent algorithms and models tailored for 2D image analysis. Transformation of the image data into a tabular format was accomplished through the application of an autoencoder [15, 16]. This step focused on reducing the dimensionality of the image data while preserving salient information. Dimensionality reduction helped to reduce computational burdens and expedite more efficient processing in subsequent analytical stages. Converting the image data into a tabular structure made it easier to integrate with diverse datasets, forming a robust foundation for various analytical techniques and models tailored for tabular data. The visualization of CT data is outlined in Fig. (2).

CT scans, clinical records, and laboratory findings are unified in a single data frame. This confluence enabled a comprehensive analysis, identifying correlations and patterns across different data types, potentially revealing insights into underlying medical conditions or outcomes.

Imputation by linear regression was used to replace missing values and to ensure dataset integrity [17]. Additionally, standardizing the dataset through L2 norm-based normalization, which scales the data to a range of 0 to 1, ensured uniformity and comparability across different features. This step effectively addressed the issue of varying scales in the dataset, promoting unbiased analyses and enhancing the interpretability and reliability of subsequent models and findings.

2.3. Data Imputation

In the realm of machine learning, medical datasets frequently contain missing values arising from a variety of factors such as equipment malfunction, incomplete patient records, data corruption, and differences in data recording sources [x+1]. These data gaps can manifest across diverse types of medical information, encompassing imaging scans, clinical measurements, and laboratory tests. Given the pivotal role of medical data in machine learning models for decision-making in healthcare settings, addressing missing values becomes imperative to uphold the reliability and accuracy of analyses and predictions. Inadequate handling of missing data can introduce biases, lead to erroneous conclusions, and potentially impact patient care adversely. Therefore, employing effective strategies for imputing missing values is crucial to maintain the integrity and usefulness of medical datasets in machine learning applications.

Imputation methods are vital for addressing missing values in medical datasets, with linear regression-based imputation being particularly noteworthy for its ability to capture relationships between variables and predict missing values based on observed data [x+2]. In medical datasets, where variables are often interconnected, linear regression provides a systematic approach to imputing missing values while preserving the underlying data structure.

Linear regression for imputation involves fitting a linear equation to the observed data, where the relationship between independent variables (features) and dependent variable (target) is represented as:

y = β ₀+β₁x₁+β₂x₂+...+β_nx_n+ϵ

• y is the dependent variable to be predicted.

• β ₀ is the intercept term.

• β₁, β₂, ... , β_n are the coefficients of the independent variables x₁, x₂, ... , x_n

• ϵ is the error term, representing the difference between the observed and predicted values.

The model is trained using observed data without missing values, enabling it to predict missing values for the target variable based on the values of other variables in the dataset. This process completes the dataset and ensures its usability for subsequent analyses and modeling tasks. However, it is essential to validate the model's assumptions, such as linearity and homoscedasticity, to ensure the reliability of the imputed values. Overall, linear regression provides a robust framework for imputing missing values in medical datasets, contributing to the integrity and usability of the data.

[x+1] Liu, M., Li, S., Yuan, H., Ong, M. E. H., Ning, Y., Xie, F., ... & Liu, N. (2023). Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques. Artificial Intelligence in Medicine, 102587.

[x+2] Codella J, Sarker H, Chakraborty P, Ghalwash M, Yao Z, and Sow D. EXITs: An ensemble approach for imputing missing EHR data. In: 2019 IEEE International Conference on Healthcare Informatics (ICHI), IEEE, 2019, 1–3.

2.4. Random Forest

The RF algorithm represents a machine learning algorithm constructed using multiple decision trees (following the bagging concept) [18]. Notably, this algorithm offers distinct advantages over conventional tree algorithms. It exhibits enhanced resistance to noise, mitigating the impact of extraneous data points, and it avoids the issue of overfitting, thus enhancing its predictive accuracy [19, 20].

The RF algorithm, a form of machine learning, operates in two distinct stages: model creation and inference (prediction/classification) [19]

2.4.2. Inference Stage

1. Test data is input into each tree, generating decisions for each tree.

2. The decisions from each tree are aggregated using methods like majority voting or averaging to derive the final outcome.

Furthermore, for a more comprehensive understanding, an inference visualization is detailed in Fig. (3), providing a visual representation of the described process. This illustration visually depicts the flow and aggregation of decisions in the RF algorithm during the inference stage.

2.5. Model Evaluation

This study meticulously evaluates the performance of various experimental scenarios by employing a range of standard metrics tailored for classification tasks. These metrics, encompassing accuracy, F1 score, precision, recall, sensitivity, specificity, and AUC (Area Under the ROC Curve), collectively provide a robust assessment of the model's predictive capabilities, enabling a comprehensive analysis of its effectiveness across diverse experimental setups. Notably, the dataset was partitioned into development and validation subsets, with metrics generated solely using the validation data to ensure unbiased evaluation.

Accuracy serves as a fundamental metric, measuring the model's ability to correctly classify data and providing a foundational understanding of its overall performance. Meanwhile, the F1 score offers a nuanced perspective on the balance between precision and recall, offering insights into the model's ability to manage false positives and false negatives.

AUC, represented by the Area Under the Receiver Operating Characteristic (ROC) Curve, quantifies the model's ability to discriminate between classes across various thresholds. Higher AUC values indicate better discrimination performance, with 1 representing perfect classification and 0.5 indicating random classification. This metric is particularly valuable for evaluating the overall performance of the model irrespective of class imbalance or threshold selection.

Moreover, the selection of metrics depends on the specific objectives and characteristics of the task under study. Precision and recall become crucial metrics in scenarios where the costs associated with false positives or false negatives vary significantly. Similarly, sensitivity and specificity play vital roles in evaluating the model's ability to differentiate between different classes, particularly in scenarios characterized by class imbalance.

By integrating these diverse metrics, including accuracy, F1 score, precision, recall, sensitivity, specificity, and AUC, into the evaluation framework, researchers gain a thorough understanding of the model's performance nuances under varying experimental conditions. Such meticulous assessment, conducted solely on the validation data, not only guides decisions concerning the model's deployment and optimization but also underscores its relevance in real-world applications where precise classification is paramount.

2.6. Experiment

The detailed process of the experiment is outlined in Fig. (4). The initial phase involves loading the pre-processed clean dataset. Subsequently, the dataset is bifurcated into two main segments: the development and validation datasets. The validation data subset is specifically extracted from the primary dataset, spanning the timeframe from 2022 to 2023, ensuring the model's adaptability and performance on more recent data.

Following dataset segmentation, the core phase involves constructing an RF-based machine-learning model. The evaluation of the developed models is performed based on predefined metrics. This assessment is intended to gauge the model's performance, compare predictions, and validate the accuracy and robustness of the machine learning models generated. These metrics serve as critical benchmarks to assess the predictive accuracy, generalizability, and overall reliability of the constructed models in this experimental framework.

To further evaluate the model's effectiveness, we aimed to integrate an alternative architecture derived from the research conducted by Bacchi et al. [21]. Their study employed a convolutional neural network (CNN) for predicting functional outcomes following thrombolysis, notably excluding laboratory values from its evaluation criteria. This contrast in methodology prompts a reconsideration of the model's construction and lends insight into the potential impact of excluding certain variables in predictive models.

Furthermore, performance assessments for both RF and CNN models were generated in three experimental scenarios: CT, CT + Clinic, and CT + Clinic + Laboratory. In this research, we deliberately avoided parameter tuning to ensure a focus on the models' inherent capabilities without introducing potential biases from extensive parameter adjustments. By refraining from parameter tuning, we aimed to provide a transparent and unbiased evaluation of the model's performance based on their default configurations, ensuring robust and reflective results.

3.2. Feature Analysis

In this study, we employed the Gini Impurity method to conduct a comprehensive analysis of variable importance, aimed at identifying key predictors significantly influencing the Δ NIHSS outcome [22]. This method allows for the quantification of each variable's contribution to impurity reduction within the decision tree's sub-nodes, providing insights into their relative importance.

As depicted in Fig. (5), the outcomes of this ranking process unveil pivotal insights into the predictive power of various factors. Fibrinogen emerges as the foremost predictor, exhibiting a substantial score surpassing 0.175, indicative of its profound impact on Δ NIHSS. Alongside, APTT demonstrates significant contribution, with scores ranging from 0.125 to 0.150, highlighting its relevance in outcome prediction.

Moreover, Diastolic Blood Pressure and CT Images showcase notable importance, with scores ranging from 0.10 to 0.125, closely followed by PT and RBG within the same range. These findings underscore the multifaceted nature of predictive variables, each playing a discernible role in shaping the outcome.

Of particular interest are factors like systolic blood pressure, age, and stroke onset, which exhibit intermediate importance, presenting scores within the range of 0.05 to 0.10. While their influence is not as pronounced as the aforementioned variables, they nonetheless contribute significantly to the predictive model.

In conclusion, our analysis utilizing the Gini Impurity method has provided valuable insights into the predictors significantly influencing the Δ NIHSS outcome. Fibrinogen emerges as a pivotal predictor, followed by APTT, Diastolic Blood Pressure, CT Images, PT, and RBG, each playing a notable role in outcome prediction. Additionally, factors such as systolic blood pressure, age, and stroke onset exhibit intermediate importance, further enriching our understanding of the predictive landscape. This comprehensive examination underscores the complex interplay of various variables in forecasting Δ NIHSS, emphasizing the need for a nuanced approach to predictive modelling in stroke management. By elucidating the diverse degrees of influence wielded by different variables, our findings pave the way for enhanced predictive accuracy and improved patient outcomes in clinical settings.

3.3. Classification of Δ NIHSS

The evaluation of the Random Forest (RF) model's performance reveals intriguing trends across different scenarios. Notably, there is a consistent improvement in accuracy, with a gradual enhancement observed from the CT scenario's initial accuracy of 0.71 to 0.71 in the CT + Clinic scenario, culminating in a peak accuracy of 0.75 in the CT + Clinic + Lab scenario. Concurrently, the Area Under the Curve (AUC) scores show a similar progression, with values of 0.29, 0.62, and 0.72 for the CT, CT + Clinic, and CT + Clinic + Lab scenarios, respectively. This sequential improvement underscores the synergistic effect of integrating clinic and laboratory data, as elucidated in Table 3.

Furthermore, the RF model demonstrates notable proficiency in distinguishing negative instances compared to positive ones, as evidenced by its high specificity of 0.88 alongside a sensitivity of 0.43. This capability underscores the model's robustness in identifying cases where the outcome is not observed, thus contributing to its overall predictive prowess.

In conclusion, our comparative analysis underscores the importance of considering architectural nuances in model selection for predictive tasks. The RF model showcases consistent and reliable performance, particularly beneficial in scenarios where interpretability and stability are paramount. Conversely, while the CNN model may offer potential in certain contexts, its performance variability and lower accuracy suggest a need for further refinement or exploration of alternative architectures. By recognizing these differences, researchers can make informed decisions regarding model selection, ultimately enhancing predictive accuracy and efficacy in clinical applications.

In contrast to the performance trends observed with the RF architecture, the CNN model, implemented from Bacchi’s architecture, yields diverse outcomes, as outlined in Table 4. Initially, for the CT scenario, the CNN model achieves a comparatively lower accuracy of 0.42. Although there's a marginal improvement in accuracy with the incorporation of clinic or laboratory data, reaching 0.54 for CT + Clinic and 0.58 for CT + Clinic + Lab, the CNN model still falls short when compared to the RF model.

Moreover, the CNN model exhibits distinctive patterns in various metrics. While specificity demonstrates an increasing trend across the scenarios, other key metrics such as AUC, F1 scores, precision, and sensitivity show a declining trend. This divergence in performance metrics underscores the nuanced differences between the RF and CNN architectures in their predictive capabilities.