The predictive Intensive Care Unit (ICU) scoring system plays an important role in ICU management for its capability of predicting important outcomes, especially mortality. There are many scoring systems that have been developed and used in the ICU. These scoring systems are primarily based on the structured clinical data contained in the electronic health record (EHR), which may suffer the loss of the important clinical information contained in the narratives and images. In this work, we build a deep learning based survival prediction model with multi-modality data to predict ICU-mortality. Four sets of features are investigated: (1) physiological measurements of Simplified Acute Physiology Score (SAPS) II, (2) common thorax diseases pre-defined by radiologists, (3) BERT-based text representations, and (4) chest X-ray image features. We use the Medical Information Mart for Intensive Care IV (MIMIC-IV) dataset to evaluate the proposed model. Our model achieves the average C-index of 0.7847 (95% confidence interval, 0.7625-0.8068), which substantially exceeds that of the baseline with SAPS-II features (0.7477 (0.7238-0.7716)). Ablation studies further demonstrate the contributions of pre-defined labels (2.12%), text features (2.68%), and image features (2.96%). Our model achieves a higher average C-index than the traditional machine learning methods under the same feature fusion setting, which suggests that the deep learning methods can outperform the traditional machine learning methods in ICU-mortality prediction. These results highlight the potential of deep learning models with multimodal information to enhance ICU-mortality prediction. We make our work publicly available at https://github.com/bionlplab/mimic-icu-mortality.

The intensive care units (ICUs) are among the most expensive and essential parts of all hospitals for extremely ill patients. This study aims to predict mortality and explore the crucial factors affecting it. Generally, in the health care systems, having a fast and precise ICU mortality prediction for patients plays a key role in care quality, resulting in reduced costs and improved survival chances of the patients. In this study, we used a medical dataset, including patients\' demographic details, underlying diseases, laboratory disorder, and LOS. Since accurate estimates are required to have optimal results, various data pre-processings as the initial steps are used here. Besides, machine learning models are employed to predict the risk of mortality ICU discharge. For AdaBoost model, these measures are considered AUC= 0.966, sensitivity (recall) = 87.88%, Kappa=0.859, F-measure = 89.23% making it, AdaBoost, accounts for the highest rate. Our model outperforms other comparison models by using various scenarios of data processing. The obtained results demonstrate that the high mortality can be caused by underlying diseases such as diabetes mellitus and high blood pressure, moderate Pulmonary Embolism Wells Score risk, platelet blood count less than 100000 (mcl), hypertension (HTN), high level of Bilirubin, smoking, and GCS level between 6 and 9.