PDF(Pubmed)
Abstract：
The process of reconstructing underlying cortical and subcortical electrical activities from Electroencephalography (EEG) or Magnetoencephalography (MEG) recordings is called Electrophysiological Source Imaging (ESI). Given the complementarity between EEG and MEG in measuring radial and tangential cortical sources, combined EEG/MEG is considered beneficial in improving the reconstruction performance of ESI algorithms. Traditional algorithms mainly emphasize incorporating predesigned neurophysiological priors to solve the ESI problem. Deep learning frameworks aim to directly learn the mapping from scalp EEG/MEG measurements to the underlying brain source activities in a data-driven manner, demonstrating superior performance compared to traditional methods. However, most of the existing deep learning approaches for the ESI problem are performed on a single modality of EEG or MEG, meaning the complementarity of these two modalities has not been fully utilized. How to fuse the EEG and MEG in a more principled manner under the deep learning paradigm remains a challenging question. This study develops a Multi-Modal Deep Fusion (MMDF) framework using Attention Neural Networks (ANN) to fully leverage the complementary information between EEG and MEG for solving the ESI inverse problem, which is termed as MMDF-ANN. Specifically, our proposed brain source imaging approach consists of four phases, including feature extraction, weight generation, deep feature fusion, and source mapping. Our experimental results on both synthetic dataset and real dataset demonstrated that using a fusion of EEG and MEG can significantly improve the source localization accuracy compared to using a single-modality of EEG or MEG. Compared to the benchmark algorithms, MMDF-ANN demonstrated good stability when reconstructing sources with extended activation areas and situations of EEG/MEG measurements with a low signal-to-noise ratio.
摘要：
从脑电图（EEG）或脑磁图（MEG）记录重建潜在的皮层和皮层下电活动的过程称为电生理源成像（ESI）。考虑到EEG和MEG在测量径向和切向皮质来源时的互补性，组合EEG/MEG被认为有利于提高ESI算法的重建性能。传统算法主要强调结合预先设计的神经生理学先验来解决ESI问题。深度学习框架旨在以数据驱动的方式直接学习从头皮EEG/MEG测量到潜在脑源活动的映射。与传统方法相比，表现出卓越的性能。然而,大多数现有的ESI问题的深度学习方法都是在单一模式的EEG或MEG上执行的，这意味着这两种模式的互补性没有得到充分利用。如何在深度学习范式下以更有原则的方式融合EEG和MEG仍然是一个具有挑战性的问题。本研究使用注意力神经网络（ANN）开发了一种多模态深度融合（MMDF）框架，以充分利用EEG和MEG之间的互补信息来解决ESI反问题。它被称为MMDF-ANN。具体来说,我们提出的脑源成像方法包括四个阶段，包括特征提取，重量生成，深层特征融合，和源映射。我们在合成数据集和真实数据集上的实验结果表明，与使用单模态的EEG或MEG相比，使用EEG和MEG的融合可以显着提高源定位精度。与基准算法相比，MMDF-ANN在重建具有扩展激活区域的源以及具有低信噪比的EEG/MEG测量情况时表现出良好的稳定性。
