PDF(Pubmed)
Abstract：
Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.
摘要：
手势是一种自然而直观的交流形式，并且将这种通信方法集成到机器人系统中具有改善人机协作的巨大潜力。运动神经科学的最新进展集中在从也称为运动原语的协同作用中复制人手运动。协同作用,运动的基本组成部分，作为中枢神经系统适应的潜在策略来产生和控制运动。确定协同作用如何促进运动可以帮助机器人的灵巧控制，外骨骼,并将其应用于康复。在本文中,通过单个RGB相机记录了33个静态手势，并通过MediaPipe框架实时识别出参与者用惯用手做出各种姿势。假设手掌张开作为初始姿势，从所有这些手势获得均匀的关节角速度。通过应用降维方法，从这些关节角速度中获得了运动学协同作用。可以解释98％的运动变化的运动学协同作用被用来使用凸优化来重建新的手势。重建的手势和选定的运动学协同作用被转换到人形机器人上，Mitra,实时,参与者展示了各种手势。结果表明，通过仅使用很少的运动学协同，可以生成各种手势，准确率为95.7%。此外，利用低维协同控制高维末端执行器有望实现近乎自然的人机协作。
