基于关键词识别的语音到手势的转换

摘要

This paper proposes a method to realize a speech-to-gesture conversion for communication between speech impediments and healthy people. The keyword spotting is employed to recognize the key words from input speech sig-nals. At the same time, the three dimensional gesture models of keywords are built by 3D modeling technology according to the“Chinese sign language”. The speech-to-gesture conversion is finally realized by playing the corresponding 3D ges-tures with OpenGL from the results of keyword spotting. Tests show that the realized keyword spotting achieves 90.1%of average recognition rate on letters and numbers. The converted gestures obtain 4.4 of mean opinion score. Therefore the proposed method can be applied to the communications between normal persons and speech impediments.%针对言语障碍者与正常人的交流问题，提出了一种利用关键词识别技术实现语音到手势转换的方法。首先，对采集到的语音信号，运用关键词识别技术识别出关键词。同时，根据《中国手语》，采用三维建模技术建立关键词对应的三维手势模型。最后，利用OpenGL播放识别出的关键词对应的三维手势模型，从而实现了语音到手势的转换。实验结果表明，字母和数字的语音关键词的平均识别率达到90.1%，转换后的手势平均MOS（Mean Opinion Score）得分为4.4分，能够应用于正常人与言语障碍者的交流。