PDF(Pubmed)
Abstract：
Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on some classification tasks. This paper complements existing research by investigating how these techniques influence classification performance and computation costs compared to full fine-tuning. We focus specifically on multilingual text classification tasks (genre, framing, and persuasion techniques detection; with different input lengths, number of predicted classes and classification difficulty), some of which have limited training data. In addition, we conduct in-depth analyses of their efficacy across different training scenarios (training on the original multilingual data; on the translations into English; and on a subset of English-only data) and different languages. Our findings provide valuable insights into the applicability of parameter-efficient fine-tuning techniques, particularly for multilabel classification and non-parallel multilingual tasks which are aimed at analysing input texts of varying length.
摘要：
适配器和低排序自适应（LoRA）是参数有效的微调技术，旨在使语言模型的训练更加有效。先前的结果表明，这些方法甚至可以提高某些分类任务的性能。本文通过研究与全面微调相比，这些技术如何影响分类性能和计算成本来补充现有研究。我们特别关注多语言文本分类任务(流派，框架,和说服技术检测；不同的输入长度，预测类的数量和分类难度)，其中一些训练数据有限。此外,我们在不同的训练场景(对原始多语言数据的训练;对翻译成英语的翻译;以及对只有英语的数据的子集)和不同的语言进行深入分析。我们的发现为参数有效的微调技术的适用性提供了有价值的见解，特别是多标签分类和非并行多语言任务，旨在分析不同长度的输入文本。
