Abstract：
Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.
摘要：
睡眠分期是睡眠质量测量和睡眠障碍诊断的基本评估。尽管当前的深度学习方法已经成功地集成了多模式睡眠信号，提高自动睡眠分期的准确性，某些挑战依然存在,如下：1)优化多模态信息互补的利用，2)有效提取睡眠信息的长程和短程时间特征,和3)解决睡眠数据中的类不平衡问题。为了应对这些挑战，本文提出了一种双流编码解码器网络，名为TSEDSleepNet，这是由深度敏感注意和自动多模态融合（DSA2F）框架的启发。在TSEDSleepNet中，双流编码器用于提取眼电图（EOG）和脑电图（EEG）信号的多尺度特征。利用自我注意机制来融合多尺度特征，生成多模态显著特征。随后，采用较粗尺度构造模块（CSCM）从多尺度特征和显著特征中提取和构造多分辨率特征。此后,变换器模块被应用于从多分辨率特征捕获长程和短程时间特征。最后,用低层细节恢复长程和短程时间特征，并映射到预测的分类结果。此外,Lovász损失函数用于缓解睡眠数据集中的类不平衡问题。我们提出的方法在Sleep-EDF-39和Sleep-EDF-153数据集上进行了测试，分类准确率分别为88.9%和85.2%，Macro-F1评分分别为84.8%和79.7%，分别,从而优于传统的基线模型。这些结果突出了所提出的方法在融合多模态信息方面的功效。该方法具有作为诊断睡眠障碍的辅助工具的应用潜力。
