Abstract：
Multi-modal medical image (MI) fusion assists in generating collaboration images collecting complement features through the distinct images of several conditions. The images help physicians to diagnose disease accurately. Hence, this research proposes a novel multi-modal MI fusion modal named guided filter-based interactive multi-scale and multi-modal transformer (Trans-IMSM) fusion approach to develop high-quality computed tomography-magnetic resonance imaging (CT-MRI) fused images for brain tumor detection. This research utilizes the CT and MRI brain scan dataset to gather the input CT and MRI images. At first, the data preprocessing is carried out to preprocess these input images to improve the image quality and generalization ability for further analysis. Then, these preprocessed CT and MRI are decomposed into detail and base components utilizing the guided filter-based MI decomposition approach. This approach involves two phases: such as acquiring the image guidance and decomposing the images utilizing the guided filter. A canny operator is employed to acquire the image guidance comprising robust edge for CT and MRI images, and the guided filter is applied to decompose the guidance and preprocessed images. Then, by applying the Trans-IMSM model, fuse the detail components, while a weighting approach is used for the base components. The fused detail and base components are subsequently processed through a gated fusion and reconstruction network, and the final fused images for brain tumor detection are generated. Extensive tests are carried out to compute the Trans-IMSM method\'s efficacy. The evaluation results demonstrated the robustness and effectiveness, achieving an accuracy of 98.64% and an SSIM of 0.94.
摘要：
多模式医学图像（MI）融合有助于生成协作图像，通过几种条件的不同图像收集补充特征。这些图像有助于医生准确诊断疾病。因此，这项研究提出了一种新颖的多模态MI融合模式，称为基于引导滤波器的交互式多尺度和多模态变压器（Trans-IMSM）融合方法，以开发高质量的计算机断层扫描-磁共振成像（CT-MRI）融合图像用于脑肿瘤检测。这项研究利用CT和MRI脑部扫描数据集来收集输入的CT和MRI图像。起初,对这些输入图像进行数据预处理,以提高图像质量和泛化能力,便于进一步分析。然后，这些预处理的CT和MRI使用基于引导滤波器的MI分解方法分解为细节和基本组件。该方法涉及两个阶段:诸如获取图像引导和利用引导滤波器分解图像。采用canny算子来获取包括CT和MRI图像的鲁棒边缘的图像引导，并应用引导滤波器对引导图像和预处理图像进行分解。然后，通过应用Trans-IMSM模型，融合细节组件，而基础组件使用加权方法。融合的细节和基础组件随后通过门控融合和重建网络进行处理，并生成用于脑肿瘤检测的最终融合图像。进行了广泛的测试以计算跨IMSM方法的功效。评价结果证明了该方法的鲁棒性和有效性,达到98.64%的精度和0.94的SSIM。
