Abstract：
Fusion of multimodal medical data provides multifaceted, disease-relevant information for diagnosis or prognosis prediction modeling. Traditional fusion strategies such as feature concatenation often fail to learn hidden complementary and discriminative manifestations from high-dimensional multimodal data. To this end, we proposed a methodology for the integration of multimodality medical data by matching their moments in a latent space, where the hidden, shared information of multimodal data is gradually learned by optimization with multiple feature collinearity and correlation constrains. We first obtained the multimodal hidden representations by learning mappings between the original domain and shared latent space. Within this shared space, we utilized several relational regularizations, including data attribute preservation, feature collinearity and feature-task correlation, to encourage learning of the underlying associations inherent in multimodal data. The fused multimodal latent features were finally fed to a logistic regression classifier for diagnostic prediction. Extensive evaluations on three independent clinical datasets have demonstrated the effectiveness of the proposed method in fusing multimodal data for medical prediction modeling.
摘要：
多模式医疗数据的融合提供了多方面的，用于诊断或预后预测建模的疾病相关信息。传统的融合策略，如特征级联，往往无法从高维多模态数据中学习隐藏的互补和判别表现。为此,我们提出了一种通过在潜在空间中匹配多模态医学数据的方法，隐藏的地方，多模态数据的共享信息通过多特征共线性和相关性约束的优化逐渐学习。我们首先通过学习原始域和共享潜在空间之间的映射获得了多模态隐藏表示。在这个共享空间中，我们利用了几个关系正则化，包括数据属性保存，特征共线性和特征-任务相关性，鼓励学习多模态数据固有的潜在关联。最终将融合的多模态潜在特征输入逻辑回归分类器进行诊断预测。对三个独立的临床数据集的广泛评估已经证明了所提出的方法在融合用于医学预测建模的多模态数据方面的有效性。
.
