Energy minimization for extracting mouth curves in a facial image

摘要

A system for extracting the mouth shape of a speaker in digital images is presented in this paper. A deformable template consisting of 2-degree polynomials is used to model the mouth outlines. The template is associated with an energy function that measures how well the model matches the mouth curves in the image. Minimization of the energy function is carried out by Powell's method in multidimensions. When applied to a variety of facial images, the method produced excellent results. Existence of facial hair and facial tilt did not affect the performance of the system. The applicability of the method seems promising in the areas of automatic speech recognition (lipreading), visual speech synthesis and speech-assisted video processing.