METHODS: Considering that it is hard to distinguish MVD-related structures, especially for nerve and vessels with similar topology, we design a multimodal segmentation network with a shared encoder-dual decoder structure and propose a clinical knowledge-driven distillation scheme, allowing reliable knowledge transferred from each decoder to the other. Besides, we introduce a class-wise contrastive module to learn the discriminative representations by maximizing the distance among classes across modalities. Then, a projected topological loss based on persistent homology is proposed to constrain topological continuity.
RESULTS: We evaluate the performance of our method on in-house dataset consisting of 100 paired HR-T2WI and 3D TOF-MRA volumes. Experiments indicate that our model outperforms the SOTA in DSC by 1.9% for artery, 3.3% for vein and 0.5% for nerve. Visualization results show our method attains improved continuity and less breakage, which is also consistent with intraoperative images.
CONCLUSIONS: Our method can comprehensively extract the distinct features from multimodal data to segment the MVD-related key structures and preserve the topological continuity, allowing surgeons precisely perceiving the patient-specific target anatomy and substantially reducing the workload of surgeons in the preoperative planning stage. Our resources will be publicly available at https://github.com/JaronTu/Multimodal_MVD_Seg .
方法:考虑到很难区分与MVD相关的结构,特别是对于具有相似拓扑结构的神经和血管,我们设计了一个具有共享编码器-双解码器结构的多模态分割网络,并提出了一种临床知识驱动的蒸馏方案,允许可靠的知识从每个解码器转移到另一个。此外,我们引入了一个类对比模块,通过最大化跨模态的类之间的距离来学习判别表示。然后,提出了一种基于持续同源性的投影拓扑损失来约束拓扑连续性。
结果:我们在由100个配对的HR-T2WI和3DTOF-MRA卷组成的内部数据集上评估了我们方法的性能。实验表明,对于动脉,我们的模型优于DSC中的SOTA1.9%,静脉为3.3%,神经为0.5%。可视化结果表明,我们的方法提高了连续性,减少了破损,这也与术中图像一致。
结论:我们的方法可以从多模态数据中全面提取不同的特征,以分割与MVD相关的关键结构并保持拓扑连续性,允许外科医生精确感知患者特定的目标解剖结构,并大大减少外科医生在术前计划阶段的工作量。我们的资源将在https://github.com/JaronTu/Multimodal_MVD_Seg上公开。