Abstract：
In this paper, we propose a multi-task learning (MTL) network based on the label-level fusion of metadata and hand-crafted features by unsupervised clustering to generate new clustering labels as an optimization goal. We propose a MTL module (MTLM) that incorporates an attention mechanism to enable the model to learn more integrated, variable information. We propose a dynamic strategy to adjust the loss weights of different tasks, and trade off the contributions of multiple branches. Instead of feature-level fusion, we propose label-level fusion and combine the results of our proposed MTLM with the results of the image classification network to achieve better lesion prediction on multiple dermatological datasets. We verify the effectiveness of the proposed model by quantitative and qualitative measures. The MTL network using multi-modal clues and label-level fusion can yield the significant performance improvement for skin lesion classification.
摘要：
在本文中,我们提出了一种多任务学习（MTL）网络，基于标签级融合元数据和手工制作的特征，通过无监督聚类生成新的聚类标签作为优化目标。我们提出了一个MTL模块（MTLM），它包含了一种注意力机制，使模型能够学习更多的集成，可变信息。我们提出了一种动态策略来调整不同任务的损失权重，并权衡多个分支机构的贡献。而不是特征级融合，我们提出了标签级融合，并将我们提出的MTLM的结果与图像分类网络的结果相结合，以在多个皮肤病学数据集上实现更好的病变预测。我们通过定量和定性措施验证了该模型的有效性。使用多模态线索和标签级融合的MTL网络可以为皮肤病变分类产生显著的性能改进。
