PDF(Pubmed)
Abstract：
Lack of an effective early sign language learning framework for a hard-of-hearing population can have traumatic consequences, causing social isolation and unfair treatment in workplaces. Alphabet and digit detection methods have been the basic framework for early sign language learning but are restricted by performance and accuracy, making it difficult to detect signs in real life. This article proposes an improved sign language detection method for early sign language learners based on the You Only Look Once version 8.0 (YOLOv8) algorithm, referred to as the intelligent sign language detection system (iSDS), which exploits the power of deep learning to detect sign language-distinct features. The iSDS method could overcome the false positive rates and improve the accuracy as well as the speed of sign language detection. The proposed iSDS framework for early sign language learners consists of three basic steps: (i) image pixel processing to extract features that are underrepresented in the frame, (ii) inter-dependence pixel-based feature extraction using YOLOv8, (iii) web-based signer independence validation. The proposed iSDS enables faster response times and reduces misinterpretation and inference delay time. The iSDS achieved state-of-the-art performance of over 97% for precision, recall, and F1-score with the best mAP of 87%. The proposed iSDS method has several potential applications, including continuous sign language detection systems and intelligent web-based sign recognition systems.
摘要：
听力障碍人群缺乏有效的早期手语学习框架可能会造成创伤性后果，在工作场所造成社会孤立和不公平待遇。字母和数字检测方法一直是早期手语学习的基本框架，但受到性能和准确性的限制。很难在现实生活中发现迹象。本文提出了一种基于YouOnlyLookOnce8.0(YOLOv8)算法的早期手语学习者的改进手语检测方法,称为智能手语检测系统（iSDS），它利用深度学习的力量来检测手语的独特特征。iSDS方法可以克服假阳性率，提高手语检测的准确性和速度。针对早期手语学习者提出的iSDS框架包括三个基本步骤：（i）图像像素处理，以提取在框架中表现不足的特征，(ii)使用YOLOv8的相互依赖的基于像素的特征提取,(iii)基于网络的签名者独立性验证。所提出的iSDS可实现更快的响应时间，并减少错误解释和推理延迟时间。iSDS的精度达到了超过97%的最先进的性能，召回，和F1得分，最佳mAP为87％。提出的iSDS方法有几个潜在的应用，包括连续手语检测系统和基于网络的智能手语识别系统。
