BACKGROUND: The different tumor appearance of head and neck cancer across imaging modalities, scanners, and acquisition parameters accounts for the highly subjective nature of the manual tumor segmentation task. The variability of the manual contours is one of the causes of the lack of generalizability and the suboptimal performance of deep learning (DL) based tumor auto-segmentation models. Therefore, a DL-based method was developed that outputs predicted tumor probabilities for each PET-CT voxel in the form of a probability map instead of one fixed contour. The aim of this study was to show that DL-generated probability maps for tumor segmentation are clinically relevant, intuitive, and a more suitable solution to assist radiation oncologists in gross tumor volume segmentation on PET-CT images of head and neck cancer patients.
METHODS: A graphical user interface (GUI) was designed, and a prototype was developed to allow the user to interact with tumor probability maps. Furthermore, a user study was conducted where nine experts in tumor delineation interacted with the interface prototype and its functionality. The participants\' experience was assessed qualitatively and quantitatively.
RESULTS: The interviews with radiation oncologists revealed their preference for using a rainbow colormap to visualize tumor probability maps during contouring, which they found intuitive. They also appreciated the slider feature, which facilitated interaction by allowing the selection of threshold values to create single contours for editing and use as a starting point. Feedback on the prototype highlighted its excellent usability and positive integration into clinical workflows.
CONCLUSIONS: This study shows that DL-generated tumor probability maps are explainable, transparent, intuitive and a better alternative to the single output of tumor segmentation models.

BACKGROUND: Machine learning (ML) models for early identification of patients at risk of hospital-acquired urinary tract infections (HA-UTI) may enable timely and targeted preventive and therapeutic strategies. However, clinicians are often challenged in the interpretation of the predictive outcomes provided by the ML models, which often reach different performances.
OBJECTIVE: To train ML models for predicting patients at risk of HA-UTI using available data from electronic health records at the time of hospital admission. We focused on the performance of different ML models and clinical explainability.
METHODS: This retrospective study investigated patient data representing 138.560 hospital admissions in the North Denmark Region from 01.01.2017 to 31.12.2018. We extracted 51 health socio-demographic and clinical features in a full dataset and used the χ2 test in addition to expert knowledge for feature selection, resulting in two reduced datasets. Seven different ML models were trained and compared between the three datasets. We applied the SHapley Additive exPlanation (SHAP) method to support population- and patient-level explainability.
RESULTS: The best-performing ML model was a neural network based on the full dataset, reaching an area under the curve (AUC) of 0.758. The neural network was also the best-performing ML model based on the reduced datasets, reaching an AUC of 0.746. Clinical explainability was demonstrated with a SHAP summary- and forceplot.
CONCLUSIONS: Within 24h of hospital admission, the ML models were able to identify patients at risk of developing HA-UTI, providing new opportunities to develop efficient strategies for the prevention of HA-UTI. Using SHAP, we demonstrate how risk predictions can be explained at individual patient level and for the patient population in general.

Medical experts may use Artificial Intelligence (AI) systems with greater trust if these are supported by \'contextual explanations\' that let the practitioner connect system inferences to their context of use. However, their importance in improving model usage and understanding has not been extensively studied. Hence, we consider a comorbidity risk prediction scenario and focus on contexts regarding the patients\' clinical state, AI predictions about their risk of complications, and algorithmic explanations supporting the predictions. We explore how relevant information for such dimensions can be extracted from Medical guidelines to answer typical questions from clinical practitioners. We identify this as a question answering (QA) task and employ several state-of-the-art Large Language Models (LLM) to present contexts around risk prediction model inferences and evaluate their acceptability. Finally, we study the benefits of contextual explanations by building an end-to-end AI pipeline including data cohorting, AI risk modeling, post-hoc model explanations, and prototyped a visual dashboard to present the combined insights from different context dimensions and data sources, while predicting and identifying the drivers of risk of Chronic Kidney Disease (CKD) - a common type-2 diabetes (T2DM) comorbidity. All of these steps were performed in deep engagement with medical experts, including a final evaluation of the dashboard results by an expert medical panel. We show that LLMs, in particular BERT and SciBERT, can be readily deployed to extract some relevant explanations to support clinical usage. To understand the value-add of the contextual explanations, the expert panel evaluated these regarding actionable insights in the relevant clinical setting. Overall, our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case. Our findings can help improve clinicians\' usage of AI models.