発話シーンからのキーフレーム検出とキーフレームに基づく単語読唇

摘要

In this paper, we propose the new keyframe-based lip reading method which does not need the advanced registration of an utterance scene. To extract keyframe, we apply the frame subtraction method and extract frame which the difference value is the local minimum as the keyframe. We compute thirteen shape features from the five lip regions of the extracted keyframe. Then we apply a discriminant analysis to mouth shape recognition. We generate a code sequence based on a mouth shape recognition result. Moreover, in accordance with several rules, we generate candidate code sequences. Finally, we apply DP matching using two kinds of code sequence of based on keyframe and candidate, and select the similar code sequence as the result word. We set Japanese 19 words as the target. We took four speakers' utterance scene. We carried out three experiments of the keyframe extraction, the mouth shape recognition, and the word recognition. As a result, we obtained average recognition rate of 53.9%. Although there was individual difference, one speaker obtained 72.1% of the highest recognition rate.%発話認識の分野において，高い認識率を誇る音声認識に 関する研究は盛んに取り組まれている。しかし音声情報の みによる認識の場合，周囲雑音の影響を受けやすく，利用 環境が制限される問題がある。近年，高騒音環境下や公共 の場所で声を出せない場面での利用が可能なインタフェー スの一つとして，読唇に関する研究が注目されている。