The threat posed by Alzheimer\'s disease (AD) to human health has grown significantly. However, the precise diagnosis and classification of AD stages remain a challenge. Neuroimaging methods such as structural magnetic resonance imaging (sMRI) and fluorodeoxyglucose positron emission tomography (FDG-PET) have been used to diagnose and categorize AD. However, feature selection approaches that are frequently used to extract additional data from multimodal imaging are prone to errors. This paper suggests using a static pulse-coupled neural network and a Laplacian pyramid to combine sMRI and FDG-PET data. After that, the fused images are used to train the Mobile Vision Transformer (MViT), optimized with Pareto-Optimal Quantum Dynamic Optimization for Neural Architecture Search, while the fused images are augmented to avoid overfitting and then classify unfused MRI and FDG-PET images obtained from the AD Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies (OASIS) datasets into various stages of AD. The architectural hyperparameters of MViT are optimized using Quantum Dynamic Optimization, which ensures a Pareto-optimal solution. The Peak Signal-to-Noise Ratio (PSNR), the Mean Squared Error (MSE), and the Structured Similarity Indexing Method (SSIM) are used to measure the quality of the fused image. We found that the fused image was consistent in all metrics, having 0.64 SIMM, 35.60 PSNR, and 0.21 MSE for the FDG-PET image. In the classification of AD vs. cognitive normal (CN), AD vs. mild cognitive impairment (MCI), and CN vs. MCI, the precision of the proposed method is 94.73%, 92.98% and 89.36%, respectively. The sensitivity is 90. 70%, 90. 70%, and 90. 91% while the specificity is 100%, 100%, and 85. 71%, respectively, in the ADNI MRI test data.

Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect. In a functional MRI study, we presented to 34 participants (16 patients and 18 matched-controls) an ecologically-valid retelling of a continuous story, performed via speech and spontaneous gestures. We split the entire story into ten-word segments, and measured the semantic complexity for each segment with idea density, a linguistic measure that is commonly used clinically to evaluate aberrant language dysfunction at the semantic level. Per segment, the presence of numbers of gestures varied (n = 0, 1, +2). Our results suggest that, in comparison to controls, patients showed reduced activation for more complex segments in the bilateral middle frontal and inferior parietal regions. Importantly, this neural aberrance was normalized in segments presented with gestures. Thus, for the first time with a naturalistic multimodal stimulation paradigm, we show that gestures reduced group differences when processing a natural story, probably by facilitating the processing of semantically complex segments of the story in schizophrenia.

UNASSIGNED: Lung cancer is a malignant tumor, for which pulmonary nodules are considered to be significant indicators. Early recognition and timely treatment of pulmonary nodules can contribute to improving the survival rate of patients with cancer. Positron emission tomography-computed tomography (PET/CT) is a noninvasive, fusion imaging technique that can obtain both functional and structural information of lung regions. However, studies of pulmonary nodules based on computer-aided diagnosis have primarily focused on the nodule level due to a reliance on the annotation of nodules, which is superficial and unable to contribute to the actual clinical diagnosis. The aim of this study was thus to develop a fully automated classification framework for a more comprehensive assessment of pulmonary nodules in PET/CT imaging data.
UNASSIGNED: We developed a two-stage multimodal learning framework for the diagnosis of pulmonary nodules in PET/CT imaging. In this framework, Stage I focuses on pulmonary parenchyma segmentation using a pretrained U-Net and PET/CT registration. Stage II aims to extract, integrate, and recognize image-level and feature-level features by employing the three-dimensional (3D) Inception-residual net (ResNet) convolutional block attention module architecture and a dense-voting fusion mechanism.
UNASSIGNED: In the experiments, the proposed model\'s performance was comprehensively validated using a set of real clinical data, achieving mean scores of 89.98%, 89.21%, 84.75%, 93.38%, 86.83%, and 0.9227 for accuracy, precision, recall, specificity, F1 score, and area under curve values, respectively.
UNASSIGNED: This paper presents a two-stage multimodal learning approach for the automatic diagnosis of pulmonary nodules. The findings reveal that the main reason for limiting model performance is the nonsolitary property of nodules in pulmonary nodule diagnosis, providing direction for future research.

UNASSIGNED: Esthesioneuroblastoma (ENB) is an uncommon malignant sinonasal tumor. There are few data regarding ENB management, namely its treatment. We review our institute\'s experience in the treatment of ENB and evaluate survival outcomes.
UNASSIGNED: Retrospective study of patients with ENB treated between 1984-2022. A total of 20 patients were identified, 13 men and 7 women, aged between 20 and 76 years.
UNASSIGNED: Eleven patients were stage C of the modified Kadish staging system at initial presentation, 7 stage B, 1 stage A and 1 stage D. Seventeen patients underwent surgery alone or combined with adjuvant treatment (radiotherapy or chemoradiotherapy). The majority of the patients (71.4%) treated with surgery alone were stage B, whereas most of the patients (63.6%) that underwent surgery combined with adjuvant treatment were stage C. Five of the 7 patients treated with surgery alone had a locoregional recurrence. Two of the 10 patients treated with surgery followed by adjuvant treatment had relapsed, locoregionally and at a distance, respectively. One patient was treated with chemotherapy and 2 patients were treated with chemoradiotherapy and neoadjuvant chemotherapy followed by chemoradiotherapy, respectively. The recurrence and persistence rates were 35% and 15%, respectively. The median time from the end of the first treatment to recurrence was 20.9 months. Two- and 5-year overall survival rates were 83.9% and 77.9%; while progression-free survival rates were 76.7% and 61.0%, respectively.
UNASSIGNED: Sixty percent of patients were treated with a multimodal approach, which appeared to be a favorable strategy for the majority of patients.

The primary aim of this study is to assess the viability of employing multimodal radiomics techniques for distinguishing between cervical spinal cord injury and spinal cord concussion in cervical magnetic resonance imaging. This is a multicenter study involving 288 patients from a major medical center as the training group, and 75 patients from two other medical centers as the testing group. Data regarding the presence of spinal cord injury symptoms and their recovery status within 72 h were documented. These patients underwent sagittal T1-weighted and T2-weighted imaging using cervical magnetic resonance imaging. Radiomics techniques are used to help diagnose whether these patients have cervical spinal cord injury or spinal cord concussion. 1197 radiomics features were extracted for each modality of each patient. The accuracy of T1 modal in testing group is 0.773, AUC is 0.799. The accuracy of T2 modal in testing group is 0.707, AUC is 0.813. The accuracy of T1 + T2 modal in testing group is 0.800, AUC is 0.840. Our research indicates that multimodal radiomics techniques utilizing cervical magnetic resonance imaging can effectively diagnose the presence of cervical spinal cord injury or spinal cord concussion.

Cancer research encompasses data across various scales, modalities, and resolutions, from screening and diagnostic imaging to digitized histopathology slides to various types of molecular data and clinical records. The integration of these diverse data types for personalized cancer care and predictive modeling holds the promise of enhancing the accuracy and reliability of cancer screening, diagnosis, and treatment. Traditional analytical methods, which often focus on isolated or unimodal information, fall short of capturing the complex and heterogeneous nature of cancer data. The advent of deep neural networks has spurred the development of sophisticated multimodal data fusion techniques capable of extracting and synthesizing information from disparate sources. Among these, Graph Neural Networks (GNNs) and Transformers have emerged as powerful tools for multimodal learning, demonstrating significant success. This review presents the foundational principles of multimodal learning including oncology data modalities, taxonomy of multimodal learning, and fusion strategies. We delve into the recent advancements in GNNs and Transformers for the fusion of multimodal data in oncology, spotlighting key studies and their pivotal findings. We discuss the unique challenges of multimodal learning, such as data heterogeneity and integration complexities, alongside the opportunities it presents for a more nuanced and comprehensive understanding of cancer. Finally, we present some of the latest comprehensive multimodal pan-cancer data sources. By surveying the landscape of multimodal data integration in oncology, our goal is to underline the transformative potential of multimodal GNNs and Transformers. Through technological advancements and the methodological innovations presented in this review, we aim to chart a course for future research in this promising field. This review may be the first that highlights the current state of multimodal modeling applications in cancer using GNNs and transformers, presents comprehensive multimodal oncology data sources, and sets the stage for multimodal evolution, encouraging further exploration and development in personalized cancer care.

OBJECTIVE: Deep learning can enhance the performance of multimodal image analysis, which is known for its noninvasive attributes and complementary efficacy, in predicting axillary lymph node (ALN) metastasis. Therefore, we established a multimodal deep learning model incorporating ultrasound (US) and magnetic resonance imaging (MRI) images to predict ALN metastasis in patients with breast cancer.
METHODS: A retrospective cohort of patients with histologically confirmed breast cancer from two hospitals composed of the primary cohort (n = 465) and the external validation cohort (n = 123). All patients had undergone both preoperative US and MRI scans. After data preprocessing, three convolutional neural network models were used to analyze the US and MRI images, respectively. After integrating the US and MRI deep learning prediction results (DLUS and DLMRI, respectively), a multimodal deep learning (DLMRI+US+Clinical parameter) model was constructed. The predictive ability of the proposed model was compared to that of the DLUS, DLMRI, combined bimodal (DLMRI+US), and clinical parameter models. Evaluation was performed using the area under the receiver operating characteristic curves (AUCs) and decision curves.
RESULTS: A total of 588 patients with breast cancer participated in this study. The DLMRI+US+Clinical parameter model outperformed the alternative models, achieving the highest AUCs of 0.819 (95% confidence interval [CI] 0.734-0.903) and 0.809 (95% CI 0.723-0.895) on the internal and external validation sets, respectively. The decision curve analysis confirmed its clinical usefulness.
CONCLUSIONS: The DLMRI+US＋Clinical parameter model demonstrates the feasibility and reliability of its performance for ALN metastasis prediction in patients with breast cancer.

In Alzheimer\'s disease (AD) assessment, traditional deep learning approaches have often employed separate methodologies to handle the diverse modalities of input data. Recognizing the critical need for a cohesive and interconnected analytical framework, we propose the AD-Transformer, a novel transformer-based unified deep learning model. This innovative framework seamlessly integrates structural magnetic resonance imaging (sMRI), clinical, and genetic data from the extensive Alzheimer\'s Disease Neuroimaging Initiative (ADNI) database, encompassing 1651 subjects. By employing a Patch-CNN block, the AD-Transformer efficiently transforms image data into image tokens, while a linear projection layer adeptly converts non-image data into corresponding tokens. As the core, a transformer block learns comprehensive representations of the input data, capturing the intricate interplay between modalities. The AD-Transformer sets a new benchmark in AD diagnosis and Mild Cognitive Impairment (MCI) conversion prediction, achieving remarkable average area under curve (AUC) values of 0.993 and 0.845, respectively, surpassing those of traditional image-only models and non-unified multimodal models. Our experimental results confirmed the potential of the AD-Transformer as a potent tool in AD diagnosis and MCI conversion prediction. By providing a unified framework that jointly learns holistic representations of both image and non-image data, the AD-Transformer paves the way for more effective and precise clinical assessments, offering a clinically adaptable strategy for leveraging diverse data modalities in the battle against AD.

BACKGROUND: Prep-4-RT is a co-designed stepped-care multimodal prehabilitation program for people scheduled to receive radiotherapy for head and neck cancer (HNC). Prehabilitation, which occurs between diagnosis and treatment commencement, aims to improve a patient\'s health to reduce the incidence and severity of current and future impairments. HNC treatment can be distressing and has detrimental impacts on function and quality of life. HNC patients have increased social vulnerabilities including higher rates of socio-economic disadvantage and engagement in lifestyle habits which increase cancer risk. High levels of physical and psychological impacts of HNC treatment and increased social vulnerabilities of this population warrant investigation of optimal pathways of care, such as prehabilitation. This paper describes a research protocol to evaluate the feasibility of Prep-4-RT, which was designed to prepare HNC patients for the physical and psychological impacts of radiotherapy.
METHODS: At least sixty adult HNC patients, scheduled to receive radiotherapy (with or without chemotherapy), will be recruited over a five-month period. All participants will receive access to Prep-4-RT self-management resources. Participants identified through screening as high-risk will also be offered individualised interventions with relevant allied health professionals prior to the commencement of radiotherapy (psychologists, dietitians, speech pathologists and physiotherapists). Participants will complete evaluation surveys assessing their experiences with Prep-4-RT resources and interventions. Clinicians will also complete program evaluation surveys. Primary feasibility outcomes include adoption (uptake and intention to try) and fidelity (adherence to the specialist prehabilitation pathway). Secondary feasibility outcomes include acceptability (patient and clinician) of and satisfaction (patient) with Prep-4-RT as well as operational costs. Feasibility outcome data will be analysed using exact binomial and one-sample t tests, as appropriate.
BACKGROUND: Ethics approval has been obtained at the Peter MacCallum Cancer Centre in Melbourne, Australia. Results will be presented at national conferences and published in peer-reviewed journal(s) so that it can be accessed by clinicians involved in the care of HNC patients receiving radiotherapy. If the model of care is found to be feasible and acceptable, the transferability and scalability to other cancer centres, or for other cancer types, may be investigated.
BACKGROUND: ANZCTA (Australian New Zealand Clinical Trials Registry) ACTRN12623000770662.

Robotic surgery has emerged as a cornerstone in urological interventions, offering effectiveness and safety for patients. For anesthesiologists, this technological advancement presents a myriad of new challenges, spanning from patient selection and assessment to intraoperative dynamics and post-surgical pain management. This article aims to elucidate these challenges and provide guidance for anesthesiologists in navigating the complexities of anesthesia administration in robotic urological procedures. Through a detailed exploration of patient optimization, team coordination, intraoperative adjustments, and post-surgical care, this article serves as a valuable resource for ensuring the success of such interventions.