PDF(Pubmed)
Abstract：
Wearable sensor data analysis with persistence features generated by topological data analysis (TDA) has achieved great successes in various applications, however, it suffers from large computational and time resources for extracting topological features. In this paper, our approach utilizes knowledge distillation (KD) that involves the use of multiple teacher networks trained with the raw time-series and persistence images generated by TDA, respectively. However, direct transfer of knowledge from the teacher models utilizing different characteristics as inputs to the student model results in a knowledge gap and limited performance. To address this problem, we introduce a robust framework that integrates multimodal features from two different teachers and enables a student to learn desirable knowledge effectively. To account for statistical differences in multimodalities, entropy based constrained adaptive weighting mechanism is leveraged to automatically balance the effects of teachers and encourage the student model to adequately adopt the knowledge from two teachers. To assimilate dissimilar structural information generated by different style models for distillation, batch and channel similarities within a mini-batch are used. We demonstrate the effectiveness of the proposed method on wearable sensor data.
摘要：
具有拓扑数据分析(TDA)生成的持久性特征的可穿戴传感器数据分析在各种应用中取得了巨大的成功,然而,它遭受大量的计算和时间资源来提取拓扑特征。在本文中,我们的方法利用知识蒸馏（KD），涉及使用由TDA生成的原始时间序列和持久性图像训练的多个教师网络，分别。然而,从教师模型中直接转移知识，利用不同的特征作为学生模型的输入，会导致知识差距和有限的表现。为了解决这个问题,我们引入了一个强大的框架，该框架集成了来自两个不同教师的多模式功能，使学生能够有效地学习所需的知识。为了解释多模态的统计差异，利用基于熵的约束自适应加权机制来自动平衡教师的影响，并鼓励学生模型充分采用两位教师的知识。为了吸收不同风格模型产生的不同结构信息进行蒸馏，使用小批量中的批量和通道相似性。我们证明了该方法在可穿戴传感器数据上的有效性。
