METHODS AND APPARATUS FOR AUDIO-VISUAL SPEAKER RECOGNITION AND UTTERANCE VERIFICATION

摘要

PURPOSE: To provide a method and a device for improving a speaker recognition rate even on acoustic adverse conditions by using visual information together with corresponding audio information during a recognition process. CONSTITUTION: Concerning signals from a video compression source 2, video/ audio data are respectively passed through expandors 10 and 12 by a demultiplexer 8, video goes from a speaker face subdivision module 20 to a visual speaking feature extractor 22 and audio directly goes to an audio feature extractor 14. Data can be exchanged directly from a camera 4 or microphone 6 to the respective extractors as well. The video (audio) data are passed from a face recognition module 24 (audio speaker recognition module 16) through a reliability estimation block 26 (18) and determines a speaker while using score coupling in an identification/confirmation coupling module 30. Besides, a module 32 makes final determination from the output of a speaking confirmation module 26 based on the inputs of the extractors 22 and 14. In this case, the other technique such as feature coupling or re-scoring can be used for the determining method as well.