Automatic speechreading with application to speaker verification

摘要

Speech not only conveys linguistic information, but also characterizes the talker's identity and therefore can be used in personal authentication. While most of the speech information is contained in the acoustic channel, the lip movement during speech production also provides useful information. We investigate the effectiveness of visual speech features in a speaker verification task. We first present the visual front-end of the automatic speechreading system. We then develop a recognition engine to train and recognize sequences of visual parameters. The experimental results based on the XM2VTS database (see http://www.ee.surrey.ac.uk/research/vssp/xm2vtsdb) demonstrate that visual information is highly effective in reducing both false acceptance and false rejection rates in speaker verification tasks.