Sign language recognition

  • 文章类型: Systematic Review
    手语的分析和识别是当前活跃的研究领域,主要集中在手语识别上。各种方法在分析方法和用于信号采集的设备方面有所不同。传统方法依赖于使用运动捕捉工具计算的视频分析或空间定位数据。与这些传统的识别和分类方法相比,肌电图(EMG)信号,测量肌肉电活动,提供检测手势的潜在技术。这些基于EMG的方法由于其优点最近受到关注。这促使我们对方法进行全面的研究,方法,以及利用EMG传感器进行手形识别的项目。在本文中,我们通过文献综述概述了手语识别领域,目的是对最重要的技术进行深入的审查。这些技术在本文中根据各自的方法进行了分类。该调查讨论了基于表面肌电图(sEMG)信号的手语识别系统的进展和挑战。这些系统已显示出希望,但面临sEMG数据可变性和传感器放置等问题。多个传感器提高可靠性和准确性。机器学习,包括深度学习,是用来应对这些挑战的。基于sEMG的手语识别中常见的分类器包括SVM,ANN,CNN,KNN,HMM,还有LSTM.当SVM和神经网络被广泛使用时,随机森林和KNN在某些情况下显示出更好的性能。多层感知器神经网络在一项研究中获得了完美的准确性。CNN,经常与LSTM配对,排名第三最受欢迎的分类器,可以实现卓越的准确性,当利用肌电图和IMU数据时,高达99.6%。LSTM在处理EMG信号中的顺序依赖性方面备受推崇,使其成为手语识别系统的重要组成部分。总之,该调查强调了SVM和ANN分类器的普遍性,但也表明了随机森林和KNN等替代分类器的有效性。在基于EMG的手语识别系统中,LSTM成为捕获顺序依赖性和改进手势识别的最合适算法。
    The analysis and recognition of sign languages are currently active fields of research focused on sign recognition. Various approaches differ in terms of analysis methods and the devices used for sign acquisition. Traditional methods rely on video analysis or spatial positioning data calculated using motion capture tools. In contrast to these conventional recognition and classification approaches, electromyogram (EMG) signals, which measure muscle electrical activity, offer potential technology for detecting gestures. These EMG-based approaches have recently gained attention due to their advantages. This prompted us to conduct a comprehensive study on the methods, approaches, and projects utilizing EMG sensors for sign language handshape recognition. In this paper, we provided an overview of the sign language recognition field through a literature review, with the objective of offering an in-depth review of the most significant techniques. These techniques were categorized in this article based on their respective methodologies. The survey discussed the progress and challenges in sign language recognition systems based on surface electromyography (sEMG) signals. These systems have shown promise but face issues like sEMG data variability and sensor placement. Multiple sensors enhance reliability and accuracy. Machine learning, including deep learning, is used to address these challenges. Common classifiers in sEMG-based sign language recognition include SVM, ANN, CNN, KNN, HMM, and LSTM. While SVM and ANN are widely used, random forest and KNN have shown better performance in some cases. A multilayer perceptron neural network achieved perfect accuracy in one study. CNN, often paired with LSTM, ranks as the third most popular classifier and can achieve exceptional accuracy, reaching up to 99.6% when utilizing both EMG and IMU data. LSTM is highly regarded for handling sequential dependencies in EMG signals, making it a critical component of sign language recognition systems. In summary, the survey highlights the prevalence of SVM and ANN classifiers but also suggests the effectiveness of alternative classifiers like random forests and KNNs. LSTM emerges as the most suitable algorithm for capturing sequential dependencies and improving gesture recognition in EMG-based sign language recognition systems.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

  • 文章类型: Journal Article
    未经授权:人类使用书面语或肢体语言(动作)等语言系统相互交流,手部动作,头部手势,面部表情,嘴唇运动,还有更多。理解手语与学习自然语言一样重要。手语是聋哑障碍或残障人士的主要交流方式。没有翻译,有听觉障碍的人与其他人说话有困难。利用机器学习技术自动识别手语识别的研究最近显示出非凡的成功并取得了重大进展。这项研究的主要目的是对迄今为止通过机器学习分类器识别乌尔都语手语的所有工作进行文献综述。
    未经评估:所有的研究都是从数据库中提取的,即,PubMed,IEEE,科学直接,和谷歌学者,使用一组结构化的关键字。每一项研究都经过了适当的筛选标准,即,排除和纳入标准。在整个文献综述中,PRISMA指南得到了充分遵循和实施。
    UNASSIGNED:本文献综述包括20篇符合资格要求的研究文章。只有这些文章被选择用于额外的全文筛选,符合同行评审和研究文章的资格要求,以及在可信期刊和会议记录中发表的研究,直到2021年7月。经过其他筛选,仅包括基于乌尔都语手语的研究。该筛选的结果分为两部分;(1)乌尔都语手语可用的所有数据集的摘要。(2)总结了所有用于识别乌尔都语手语的机器学习技术。
    UNASSIGNED:我们的研究发现,只有一个公开可用的基于USL符号的数据集,其中包含图片与许多字符-,number-,或基于句子的公开数据集。还得出结论,除了支持向量机和神经网络之外,唯一分类器不会被多次使用。此外,没有研究人员选择无监督的机器学习分类器进行检测。据我们所知,这是对应用于乌尔都语手语的机器学习方法进行的第一篇文献综述。
    UNASSIGNED: Humans communicate with one another using language systems such as written words or body language (movements), hand motions, head gestures, facial expressions, lip motion, and many more. Comprehending sign language is just as crucial as learning a natural language. Sign language is the primary mode of communication for those who have a deaf or mute impairment or are disabled. Without a translator, people with auditory difficulties have difficulty speaking with other individuals. Studies in automatic recognition of sign language identification utilizing machine learning techniques have recently shown exceptional success and made significant progress. The primary objective of this research is to conduct a literature review on all the work completed on the recognition of Urdu Sign Language through machine learning classifiers to date.
    UNASSIGNED: All the studies have been extracted from databases, i.e., PubMed, IEEE, Science Direct, and Google Scholar, using a structured set of keywords. Each study has gone through proper screening criteria, i.e., exclusion and inclusion criteria. PRISMA guidelines have been followed and implemented adequately throughout this literature review.
    UNASSIGNED: This literature review comprised 20 research articles that fulfilled the eligibility requirements. Only those articles were chosen for additional full-text screening that follows eligibility requirements for peer-reviewed and research articles and studies issued in credible journals and conference proceedings until July 2021. After other screenings, only studies based on Urdu Sign language were included. The results of this screening are divided into two parts; (1) a summary of all the datasets available on Urdu Sign Language. (2) a summary of all the machine learning techniques for recognizing Urdu Sign Language.
    UNASSIGNED: Our research found that there is only one publicly-available USL sign-based dataset with pictures versus many character-, number-, or sentence-based publicly available datasets. It was also concluded that besides SVM and Neural Network, no unique classifier is used more than once. Additionally, no researcher opted for an unsupervised machine learning classifier for detection. To the best of our knowledge, this is the first literature review conducted on machine learning approaches applied to Urdu sign language.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

公众号