Stroke is a significant cause of mortality and morbidity, necessitating early predictive strategies to minimize risks. Traditional methods for evaluating patients, such as Acute Physiology and Chronic Health Evaluation (APACHE II, IV) and Simplified Acute Physiology Score III (SAPS III), have limited accuracy and interpretability. This paper proposes a novel approach: an interpretable, attention-based transformer model for early stroke mortality prediction. This model seeks to address the limitations of previous predictive models, providing both interpretability (providing clear, understandable explanations of the model) and fidelity (giving a truthful explanation of the model\'s dynamics from input to output). Furthermore, the study explores and compares fidelity and interpretability scores using Shapley values and attention-based scores to improve model explainability. The research objectives include designing an interpretable attention-based transformer model, evaluating its performance compared to existing models, and providing feature importance derived from the model.

Primary Immune Thrombocytopenia (ITP) is a rare autoimmune disease characterised by the immune-mediated destruction of peripheral blood platelets in patients leading to low platelet counts and bleeding. The diagnosis and effective management of ITP are challenging because there is no established test to confirm the disease and no biomarker with which one can predict the response to treatment and outcome. In this work, we conduct a feasibility study to check if machine learning can be applied effectively for the diagnosis of ITP using routine blood tests and demographic data in a non-acute outpatient setting. Various ML models, including Logistic Regression, Support Vector Machine, k-Nearest Neighbor, Decision Tree and Random Forest, were applied to data from the UK Adult ITP Registry and a general haematology clinic. Two different approaches were investigated: a demographic-unaware and a demographic-aware one. We conduct extensive experiments to evaluate the predictive performance of these models and approaches, as well as their bias. The results revealed that Decision Tree and Random Forest models were both superior and fair, achieving nearly perfect predictive and fairness scores, with platelet count identified as the most significant variable. Models not provided with demographic information performed better in terms of predictive accuracy but showed lower fairness scores, illustrating a trade-off between predictive performance and fairness.

Explainability is key to enhancing the trustworthiness of artificial intelligence in medicine. However, there exists a significant gap between physicians\' expectations for model explainability and the actual behavior of these models. This gap arises from the absence of a consensus on a physician-centered evaluation framework, which is needed to quantitatively assess the practical benefits that effective explainability should offer practitioners. Here, we hypothesize that superior attention maps, as a mechanism of model explanation, should align with the information that physicians focus on, potentially reducing prediction uncertainty and increasing model reliability. We employed a multimodal transformer to predict lymph node metastasis of rectal cancer using clinical data and magnetic resonance imaging. We explored how well attention maps, visualized through a state-of-the-art technique, can achieve agreement with physician understanding. Subsequently, we compared two distinct approaches for estimating uncertainty: a standalone estimation using only the variance of prediction probability, and a human-in-the-loop estimation that considers both the variance of prediction probability and the quantified agreement. Our findings revealed no significant advantage of the human-in-the-loop approach over the standalone one. In conclusion, this case study did not confirm the anticipated benefit of the explanation in enhancing model reliability. Superficial explanations could do more harm than good by misleading physicians into relying on uncertain predictions, suggesting that the current state of attention mechanisms should not be overestimated in the context of model explainability.

Early identification of patients at risk of hospital-acquired urinary tract infections (HA-UTI) enables the initiation of timely targeted preventive and therapeutic strategies. Machine learning (ML) models have shown great potential for this purpose. However, existing ML models in infection control have demonstrated poor ability to support explainability, which challenges the interpretation of the result in clinical practice, limiting the adaption of the ML models into a daily clinical routine. In this study, we developed Bayesian Network (BN) models to enable explainable assessment within 24 h of admission for risk of HA-UTI. Our dataset contained 138,250 unique hospital admissions. We included data on admission details, demographics, lifestyle factors, comorbidities, vital parameters, laboratory results, and urinary catheter. Models developed from a reduced set of five features were characterized by transparency compared to models developed from a full set of 50 features. The expert-based clinical BN model over the reduced feature space showed the highest performance (area under the curve = 0.746) compared to the naïve- and tree-augmented-naïve BN models. Moreover, models developed from expert-based knowledge were characterized by enhanced explainability.

BACKGROUND: Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration.
OBJECTIVE: This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.
METHODS: This study used a data set of 861 PSE reports from a large academic hospital\'s maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier\'s predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems.
RESULTS: The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95/126) compared to an accuracy of 66.7% (84/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight.
CONCLUSIONS: This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.

Outlier detection is a fundamental data analytics technique often used for many security applications. Numerous outlier detection techniques exist, and in most cases are used to directly identify outliers without any interaction. Typically the underlying data used is often high dimensional and complex. Even though outliers may be identified, since humans can easily grasp low dimensional spaces, it is difficult for a security expert to understand/visualize why a particular event or record has been identified as an outlier. In this paper we study the extent to which outlier detection techniques work in smaller dimensions and how well dimensional reduction techniques still enable accurate detection of outliers. This can help us to understand the extent to which data can be visualized while still retaining the intrinsic outlyingness of the outliers.

OBJECTIVE: Due to the increased risk of acute cardiac injury (ACI) and poor prognosis in cancer patients with COVID-19 infection, our aim was to develop a novel and interpretable model for predicting ACI occurrence in cancer patients with COVID-19 infection.
METHODS: This retrospective observational study screened 740 cancer patients with COVID-19 infection from December 2022 to April 2023. The least absolute shrinkage and selection operator (LASSO) regression was used for the preliminary screening of the indices. To enhance the model accuracy, we introduced an alpha index to further screen and rank the indices based on their significance. Random forest (RF) was used to construct the prediction model. The Shapley Additive Explanation (SHAP) and Local Interpretable Model-Agnostic Explanation (LIME) methods were utilized to explain the model.
RESULTS: According to the inclusion criteria, 201 cancer patients with COVID-19, including 36 variables indices, were included in the analysis. The top eight indices (albumin, lactate dehydrogenase, cystatin C, neutrophil count, creatine kinase isoenzyme, red blood cell distribution width, D-dimer and chest computed tomography) for predicting the occurrence of ACI in cancer patients with COVID-19 infection were included in the RF model. The model achieved an area under curve (AUC) of 0.940, an accuracy of 0.866, a sensitivity of 0.750 and a specificity of 0.900. The calibration curve and decision curve analysis showed good calibration and clinical practicability. SHAP results demonstrated that albumin was the most important index for predicting the occurrence of ACI. LIME results showed that the model could predict the probability of ACI in each cancer patient infected with COVID-19 individually.
CONCLUSIONS: We developed a novel machine-learning model that demonstrates high explainability and accuracy in predicting the occurrence of ACI in cancer patients with COVID-19 infection, using laboratory and imaging indices.

Physicochemical and pharmacokinetic compound profile has crucial impact on compound potency to become a future drug. Ligands with desired activity profile cannot be used for treatment if they are characterized by unfavourable physicochemical or ADMET properties. In the study, we consider metabolic stability and focus on selected subtypes of cytochrome P450 - proteins, which take part in the first phase of compound transformations in the organism. We develop a protocol for generation of new potential inhibitors of selected cytochrome isoforms. Its subsequent stages are composed of generation and assessment of new derivatives of known cytochrome inhibitors, docking and evaluation of the compound possible inhibition on the basis of the obtained ligand-protein complexes. Besides the library of new potential agents inhibiting particular cytochrome subtypes, we also prepare a graph neural network that predicts the change in activity for all modifications of the starting molecule. In addition, we perform a systematic statistical study on the influence of particular substitutions on the potential inhibition properties of generated compounds (both mono- and di-substitutions are considered), provide explanations of the inhibitory predictions and prepare an on-line visualization platform enabling manual inspection of the results. The developed methodology can greatly support the design of new cytochrome P450 inhibitors with the overarching goal of generation of new metabolically stable compounds. It enables instant evaluation of possible compound-cytochrome interactions and selection of ligands with the highest potential of possessing desired biological activity.

Chronic Ocular Diseases (COD) such as myopia, diabetic retinopathy, age-related macular degeneration, glaucoma, and cataract can affect the eye and may even lead to severe vision impairment or blindness. According to a recent World Health Organization (WHO) report on vision, at least 2.2 billion individuals worldwide suffer from vision impairment. Often, overt signs indicative of COD do not manifest until the disease has progressed to an advanced stage. However, if COD is detected early, vision impairment can be avoided by early intervention and cost-effective treatment. Ophthalmologists are trained to detect COD by examining certain minute changes in the retina, such as microaneurysms, macular edema, hemorrhages, and alterations in the blood vessels. The range of eye conditions is diverse, and each of these conditions requires a unique patient-specific treatment. Convolutional neural networks (CNNs) have demonstrated significant potential in multi-disciplinary fields, including the detection of a variety of eye diseases. In this study, we combined several preprocessing approaches with convolutional neural networks to accurately detect COD in eye fundus images. To the best of our knowledge, this is the first work that provides a qualitative analysis of preprocessing approaches for COD classification using CNN models. Experimental results demonstrate that CNNs trained on the region of interest segmented images outperform the models trained on the original input images by a substantial margin. Additionally, an ensemble of three preprocessing techniques outperformed other state-of-the-art approaches by 30% and 3%, in terms of Kappa and F 1 scores, respectively. The developed prototype has been extensively tested and can be evaluated on more comprehensive COD datasets for deployment in the clinical setup.

Heterogeneity among patients\' responses to treatment is prevalent in psychiatric disorders. Personalized medicine approaches-which involve parsing patients into subgroups better indicated for a particular treatment-could therefore improve patient outcomes and serve as a powerful tool in patient selection within clinical trials. Machine learning approaches can identify patient subgroups but are often not \"explainable\" due to the use of complex algorithms that do not mirror clinicians\' natural decision-making processes.
Here we combine two analytical approaches-Personalized Advantage Index and Bayesian Rule Lists-to identify paliperidone-indicated schizophrenia patients in a way that emphasizes model explainability. We apply these approaches retrospectively to randomized, placebo-controlled clinical trial data to identify a paliperidone-indicated subgroup of schizophrenia patients who demonstrate a larger treatment effect (outcome on treatment superior than on placebo) than that of the full randomized sample as assessed with Cohen\'s d. For this study, the outcome corresponded to a reduction in the Positive and Negative Syndrome Scale (PANSS) total score which measures positive (e.g., hallucinations, delusions), negative (e.g., blunted affect, emotional withdrawal), and general psychopathological (e.g., disturbance of volition, uncooperativeness) symptoms in schizophrenia.
Using our combined explainable AI approach to identify a subgroup more responsive to paliperidone than placebo, the treatment effect increased significantly over that of the full sample (p < 0.0001 for a one-sample t-test comparing the full sample Cohen\'s d = 0.82 and a generated distribution of subgroup Cohen\'s d\'s with mean d = 1.22, std d = 0.09). In addition, our modeling approach produces simple logical statements (if-then-else), termed a \"rule list\", to ease interpretability for clinicians. A majority of the rule lists generated from cross-validation found two general psychopathology symptoms, disturbance of volition and uncooperativeness, to predict membership in the paliperidone-indicated subgroup.
These results help to technically validate our explainable AI approach to patient selection for a clinical trial by identifying a subgroup with an improved treatment effect. With these data, the explainable rule lists also suggest that paliperidone may provide an improved therapeutic benefit for the treatment of schizophrenia patients with either of the symptoms of high disturbance of volition or high uncooperativeness.
clincialtrials.gov identifier: NCT 00,083,668; prospectively registered May 28, 2004.