Auditory-Visual Speech Recognition Error Detection using Longest Common Subsequence Matching of Vowel Sequences

摘要

Speech recognition based on only auditory information tends to be very susceptible to external noise. In this study, we investigated how visual information can be useful to improve speech recognition. We can guess a part of talking by an opponent from a shape of his/her mouth. Especially, information about vowels in opponent's talk is able to be sufficiently obtained from the lip shape. The information about an order of sequence of vowels which is found by analyzing the change of the lip shape can be used as an indicator to evaluate confidence for recognition based on an unclear voice. Using the longest common subsequence algorithm, comparison of the order of sequences of vowels resulted from the speech recognition based on voice and one from the lip reading based on version is helpful to improve confidence for the speech recognition. Studies to decrease error ratio of speech recognition allow magnifying application of a technique of speech recognition in a noisy environment.