Lip-reading device and lip-reading method

摘要

PROBLEM TO BE SOLVED: To estimate the utterance content with high accuracy by performing machine learning in consideration of facial expression characteristics in addition to the lips characteristics of the speaker, and if necessary, attributes such as the speaker's age and gender. By also considering the above, a lip-reading device and a lip-reading method capable of further improving the estimation accuracy will be provided. SOLUTION: A face detection means 22 in which an image processing unit 14 for image processing a learning target image and an evaluation target image detects a facial expression of a learning target speaker and an evaluation target speaker from the learning target image and the evaluation target image. A facial feature point detecting means 23 that detects facial feature points from each face region, a lip region extracting means 24 that extracts a lip region from the facial feature points of each face region, a face region, a facial feature point, and a speaker to be learned. The facial expression features and lip features of the learning target speaker, which is the learning target data, are extracted from the lip region, and the facial expression of the evaluation target speaker, which is the evaluation target data, is extracted from the face area, facial feature points, and lip region of the evaluation target speaker. It has a feature extraction means 25 for extracting features and lip features. [Selection diagram] Fig. 2