Molecular Property Prediction (MPP) is vital for drug discovery, crop protection, and environmental science. Over the last decades, diverse computational techniques have been developed, from using simple physical and chemical properties and molecular fingerprints in statistical models and classical machine learning to advanced deep learning approaches. In this review, we aim to distill insights from current research on employing transformer models for MPP. We analyze the currently available models and explore key questions that arise when training and fine-tuning a transformer model for MPP. These questions encompass the choice and scale of the pretraining data, optimal architecture selections, and promising pretraining objectives. Our analysis highlights areas not yet covered in current research, inviting further exploration to enhance the field\'s understanding. Additionally, we address the challenges in comparing different models, emphasizing the need for standardized data splitting and robust statistical analysis.

UNASSIGNED: Most process mining techniques are primarily automated, meaning that process analysts input information and receive output. As a result, process mining techniques function like black boxes with limited interaction options for analysts, such as simple sliders for filtering infrequent behavior. Recent research tries to break these black boxes by allowing process analysts to provide domain knowledge and guidance to process mining techniques, i.e., hybrid intelligence. Especially, in process discovery-a critical type of process mining-interactive approaches emerged. However, little research has investigated the practical application of such interactive approaches. This paper presents a case study focusing on using incremental and interactive process discovery techniques in the healthcare domain. Though healthcare presents unique challenges, such as high process execution variability and poor data quality, our case study demonstrates that an interactive process mining approach can effectively address these challenges.
UNASSIGNED:

Automatic electrocardiogram (ECG) classification provides valuable auxiliary information for assisting disease diagnosis and has received much attention in research. The success of existing classification models relies on fitting the labeled samples for every ECG type. However, in practice, well-annotated ECG datasets usually cover only limited ECG types. It thus raises an issue: conventional classification models trained with limited ECG types can only identify those ECG types that have already been observed in the training set, but fail to recognize unseen (or unknown) ECG types that exist in the wild and are not included in training data. In this work, we investigate an important problem called open-world ECG classification that can predict fine-grained observed ECG classes and identify unseen classes. Accordingly, we propose a customized method that first incorporates clinical knowledge into contrastive learning by generating \"hard negative\" samples to guide learning diagnostic ECG features (i.e., distinguishable representations), and then performs multi-hypersphere learning to learn compact ECG representations for classification. The experiment results on 12-lead ECG datasets (CPSC2018, PTB-XL, and Georgia) demonstrate that the proposed method outperforms the state-of-the-art methods. Specifically, our method achieves superior accuracy than the comparative methods on the unseen ECG class and certain seen classes. Overall, the investigated problem (i.e., open-world ECG classification) helps to draw attention to the reliability of automatic ECG diagnosis, and the proposed method is proven effective in tackling the challenges. The code and datasets are released at https://github.com/betterzhou/Open_World_ECG_Classification.

Computer-aided diagnosis (CAD) for thyroid nodules has been studied for years, yet there are still reliability and interpretability challenges due to the lack of clinically-relevant evidence. To address this issue, inspired by Thyroid Imaging Reporting and Data System (TI-RADS), we propose a novel interpretable two-branch bi-coordinate network based on multi-grained domain knowledge. First, we transform the two types of domain knowledge provided by TI-RADS, namely region-based and boundary-based knowledge, into labels at multi-grained levels: coarse-grained classification labels, and fine-grained region segmentation masks and boundary localization vectors. We combine these two labels to form the Multi-grained Domain Knowledge Representation (MG-DKR) of TI-RADS. Then we design a Two-branch Bi-coordinate network (TB2C-net) which utilizes two branches to predict MG-DKR from both Cartesian and polar images, and uses an attention-based integration module to integrate the features of the two branches for benign-malignant classification. We validated our method on a large cohort containing 3245 patients (with 3558 nodules and 6466 ultrasound images). Results show that our method achieves competitive performance with AUC of 0.93 and ACC of 0.87 compared with other state-of-the-art methods. Ablation experiment results demonstrate the effectiveness of the TB2C-net and MG-DKR, and the knowledge attention map from the integration module provides the interpretability for benign-malignant classification.

BACKGROUND: Clinical medicine offers a promising arena for applying Machine Learning (ML) models. However, despite numerous studies employing ML in medical data analysis, only a fraction have impacted clinical care. This article underscores the importance of utilising ML in medical data analysis, recognising that ML alone may not adequately capture the full complexity of clinical data, thereby advocating for the integration of medical domain knowledge in ML.
METHODS: The study conducts a comprehensive review of prior efforts in integrating medical knowledge into ML and maps these integration strategies onto the phases of the ML pipeline, encompassing data pre-processing, feature engineering, model training, and output evaluation. The study further explores the significance and impact of such integration through a case study on diabetes prediction. Here, clinical knowledge, encompassing rules, causal networks, intervals, and formulas, is integrated at each stage of the ML pipeline, resulting in a spectrum of integrated models.
RESULTS: The findings highlight the benefits of integration in terms of accuracy, interpretability, data efficiency, and adherence to clinical guidelines. In several cases, integrated models outperformed purely data-driven approaches, underscoring the potential for domain knowledge to enhance ML models through improved generalisation. In other cases, the integration was instrumental in enhancing model interpretability and ensuring conformity with established clinical guidelines. Notably, knowledge integration also proved effective in maintaining performance under limited data scenarios.
CONCLUSIONS: By illustrating various integration strategies through a clinical case study, this work provides guidance to inspire and facilitate future integration efforts. Furthermore, the study identifies the need to refine domain knowledge representation and fine-tune its contribution to the ML model as the two main challenges to integration and aims to stimulate further research in this direction.

BACKGROUND: Predicting early recurrence (ER) of hepatocellular carcinoma (HCC) accurately can guide treatment decisions and further enhance survival. Computed tomography (CT) imaging, analyzed by deep learning (DL) models combining domain knowledge, has been employed for the prediction. However, these DL models utilized late fusion, restricting the interaction between domain knowledge and images during feature extraction, thereby limiting the prediction performance and compromising decision-making interpretability.
METHODS: We propose a novel Vision Transformer (ViT)-based DL network, referred to as Dual-Style ViT (DSViT), to augment the interaction between domain knowledge and images and the effective fusion among multi-phase CT images for improving both predictive performance and interpretability. We apply the DSViT to develop pre-/post-operative models for predicting ER. Within DSViT, to balance the utilization between domain knowledge and images within DSViT, we propose an adaptive self-attention mechanism. Moreover, we present an attention-guided supervised learning module for balancing the contributions of multi-phase CT images to prediction and a domain knowledge self-supervision module for enhancing the fusion between domain knowledge and images, thereby further improving predictive performance. Finally, we provide the interpretability of the DSViT decision-making.
RESULTS: Experiments on our multi-phase data demonstrate that DSViTs surpass the existing models across multiple performance metrics and provide the decision-making interpretability. Additional validation on a publicly available dataset underscores the generalizability of DSViT.
CONCLUSIONS: The proposed DSViT can significantly improve the performance and interpretability of ER prediction, thereby fortifying the trustworthiness of artificial intelligence tool for HCC ER prediction in clinical settings.

UNASSIGNED: Manual planning of scans in clinical magnetic resonance imaging (MRI) exhibits poor accuracy, lacks consistency, and is time-consuming. Meanwhile, classical automated scan plane positioning methods that rely on certain assumptions are not accurate or stable enough, and are computationally inefficient for practical application scenarios. This study aims to develop and evaluate an effective, reliable, and accurate deep learning-based framework that incorporates prior physical knowledge for automatic head scan plane positioning in MRI.
UNASSIGNED: A deep learning-based end-to-end automated scan plane positioning framework has been developed for MRI head scans. Our model takes a three-dimensional (3D) pre-scan image input, utilizing a cascaded 3D convolutional neural network to detect anatomical landmarks from coarse to fine. And then, with the determined landmarks, accurate scan plane localization can be achieved. A multi-scale spatial information fusion module was employed to aggregate high- and low-resolution features, combined with physically meaningful point regression loss (PRL) function and direction regression loss (DRL) function. Meanwhile, we simulate complex clinical scenarios to design data augmentation strategies.
UNASSIGNED: Our proposed approach shows good performance on a clinically wide range of 229 MRI head scans, with a point-to-point absolute error (PAE) of 0.872 mm, a point-to-point relative error (PRE) of 0.10%, and an average angular error (AAE) of 0.502°, 0.381°, and 0.675° for the sagittal, transverse, and coronal planes, respectively.
UNASSIGNED: The proposed deep learning-based automated scan plane positioning shows high efficiency, accuracy and robustness when evaluated on varied clinical head MRI scans with differences in positioning, contrast, noise levels and pathologies.

Utilizing functional magnetic resonance imaging (fMRI) to model functional brain networks (FBNs) is increasingly prominent in attention-deficit/hyperactivity disorder (ADHD) research, revealing neural impact and mechanisms through the exploration of activated brain regions. However, current FBNs-based methods face two major challenges. The primary challenge stems from the limitations of existing modeling methods in accurately capturing both regional correlations and long-distance dependencies (LDDs) within the dynamic brain, thereby affecting the diagnostic accuracy of FBNs as biomarkers. Additionally, limited sample size and class imbalance also pose a challenge to the learning performance of the model. To address the issues, we propose an automated diagnostic framework, which integrates modeling, multimodal fusion, and classification into a unified process. It aims to extract representative FBNs and efficiently incorporate domain knowledge to guide ADHD classification. Our work mainly includes three-fold: 1) A multi-head attention-based region-enhancement module (MAREM) is designed to simultaneously capture regional correlations and LDDs across the entire sequence of brain activity, which facilitates the construction of representative FBNs. 2) The multimodal supplementary learning module (MSLM) is proposed to integrate domain knowledge from phenotype data with FBNs from neuroimaging data, achieving information complementarity and alleviating the problems of insufficient medical data and unbalanced sample categories. 3) An ADHD automatic diagnosis framework guided by FBNs and domain knowledge (ADF-FAD) is proposed to help doctors make more accurate decisions, which is applied to the ADHD-200 dataset to confirm its effectiveness. The results indicate that the FBNs extracted by MAREM perform well in modeling and classification. After with MSLM, the model achieves accuracy of 92.4%, 74.4%, and 80% at NYU, PU, and KKI, respectively, demonstrating its ability to effectively capture crucial information related to ADHD diagnosis. Codes are available at https://github.com/zhuimengxuebao/ADF-FAD.

Objective. Myocardial infarction (MI) is a serious cardiovascular disease that can cause irreversible damage to the heart, making early identification and treatment crucial. However, automatic MI detection and localization from an electrocardiogram (ECG) remain challenging. In this study, we propose two models, MFB-SENET and MFB-DMIL, for MI detection and localization, respectively.Approach. The MFB-SENET model is designed to detect MI, while the MFB-DMIL model is designed to localize MI. The MI localization model employs a specialized attention mechanism to integrate multi-instance learning with domain knowledge. This approach incorporates handcrafted features and introduces a new loss function called lead-loss, to improve MI localization. Grad-CAM is employed to visualize the decision-making process.Main Results.The proposed method was evaluated on the PTB and PTB-XL databases. Under the inter-patient scheme, the accuracy of MI detection and localization on the PTB database reached 93.88% and 67.17%, respectively. The accuracy of MI detection and localization on the PTB-XL database were 94.89% and 85.83%, respectively.Significance. Our method achieved comparable or better performance than other state-of-the-art algorithms. The proposed method combined deep learning and medical domain knowledge, demonstrates effectiveness and reliability, holding promise as an efficient MI diagnostic tool to assist physicians in formulating accurate diagnoses.

We propose a Multi-scale, domain knowledge-Guided Attention model (MGA-Net) for a weakly supervised problem - disease diagnosis with only coarse scan-level labels. The use of guided attention models encourages the deep learning-based diagnosis model to focus on the area of interests (in our case, lung parenchyma), at different resolutions, in an end-to-end manner. The research interest is to diagnose subjects with idiopathic pulmonary fibrosis (IPF) among subjects with interstitial lung disease (ILD) using an axial chest high resolution computed tomography (HRCT) scan. Our dataset contains 279 IPF patients and 423 non-IPF ILD patients. The network\'s performance was evaluated by the area under the receiver operating characteristic curve (AUC) with standard errors (SE) using stratified five-fold cross validation. We observe that without attention modules, the IPF diagnosis model performs unsatisfactorily (AUC±SE =0.690 ± 0.194); by including unguided attention module, the IPF diagnosis model reaches satisfactory performance (AUC±SE =0.956±0.040), but lack explainability; when including only guided high- or medium- resolution attention, the learned attention maps highlight the lung areas but the AUC decreases; when including both high- and medium- resolution attention, the model reaches the highest AUC among all experiments (AUC± SE =0.971 ±0.021) and the estimated attention maps concentrate on the regions of interests for this task. Our results suggest that, for a weakly supervised task, MGA-Net can utilize the population-level domain knowledge to guide the training of the network in an end-to-end manner, which increases both model accuracy and explainability.