PDF(Pubmed)
Abstract：
Natural language processing (NLP) tasks can be addressed with several deep learning architectures, and many different approaches have proven to be efficient. This study aims to briefly summarize the use cases for NLP tasks along with the main architectures. This research presents transformer-based solutions for NLP tasks such as Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-Training (GPT) architectures. To achieve that, we conducted a step-by-step process in the review strategy: identify the recent studies that include Transformers, apply filters to extract the most consistent studies, identify and define inclusion and exclusion criteria, assess the strategy proposed in each study, and finally discuss the methods and architectures presented in the resulting articles. These steps facilitated the systematic summarization and comparative analysis of NLP applications based on Transformer architectures. The primary focus is the current state of the NLP domain, particularly regarding its applications, language models, and data set types. The results provide insights into the challenges encountered in this research domain.
摘要：
自然语言处理（NLP）任务可以通过多种深度学习架构来解决。许多不同的方法被证明是有效的。本研究旨在简要总结NLP任务的用例以及主要架构。这项研究提出了NLP任务的基于变压器的解决方案，例如变压器的双向编码器表示（BERT），和创成式预培训(GPT)架构。要做到这一点,我们在审查策略中进行了逐步的过程：确定包括变形金刚在内的最新研究，应用过滤器来提取最一致的研究，确定和定义纳入和排除标准，评估每一项研究中提出的策略，最后讨论了所产生的文章中提出的方法和体系结构。这些步骤促进了基于Transformer架构的NLP应用程序的系统总结和比较分析。主要焦点是NLP域的当前状态，特别是关于它的应用，语言模型，和数据集类型。研究结果为该研究领域遇到的挑战提供了见解。
