Analysis of Mouth Shape Deformation Rate for Generation of Japanese Utterance Images Automatically

摘要

We have been doing research on machine lip-reading. In the process of the study, we proposed a reproduction method of utterance images without voice from Japanese kana. Firstly, sequence of codes that is called "Mouth Shapes Sequence Code" is generated. The Mouth Shapes Sequence Code expresses the order of mouth shapes when a Japanese word is uttered. The utterance images are generated by using the mouth images corresponding to the Mouth Shapes Sequence Code and the deformed mouth shape images generated with morphing. However, the deformation rate of the mouth shapes has been decided from the real utterance images experimentally. Therefore, there were cases in which the utterance images with the sense of incongruity about mouth shape deformation were generated. In this paper, the deformation rate of the mouth shapes is analyzed using real utterance images captured by a high-speed camera, and we propose a generation method of utterance images based on the results. Finally, the mean opinion score of the subjects are shown, and we evaluate the effectiveness of proposed method.