Facial expression analysis and synthesis for model based coding.

摘要

The growing interest in video communication in areas such as education, entertainment, and business (videoconferencing) makes video compression an inexhaustible research topic. The greater the prior knowledge of objects there is in the image explored, the less the amount of information to be transmitted. Model-based coding is just one instance of this principle, which has attracted many researchers both in computer vision and in computer graphics, to attempt to build a framework, especially for the human face. After a decade of efforts, it is still a challenging research topic. The primary difficulties are: (1) how to model a face automatically, (2) how to model and track the expressions accurately, and (3) how to make the synthesized face look realistic.; In this thesis, four new approaches to deal with the above three issues are proposed. They are: (1) Automatic face modeling based on two views of a face: unlike the traditional methods of using the laser scanner and the techniques of shape from shading or shape from contour, a new approach to automatically generate a 3D facial model is presented, in which an individual 3D facial model is constructed by fitting a generic head model to front and side views of a person's head. (2) Facial feature shape detection: a color-based deformable template feature detection with active tracking is proposed. The method is the first attempt to incorporate the facial feature detection with tracking by an active camera. (3) Physics-based coarse-to-fine model adaptation: an energy minimization adaptation method is proposed for tracking the facial expression accurately. There are two steps consisting of physics-based dynamic mesh matching and energy-oriented mesh fitting. This method overcomes the convergence problem of the numerical solution for the elastic motion. (4) Active texture detection, compression, and synthesis for producing a realistic face: a partial active texture up date scheme is proposed, which deals with not only the facial organ features, but also the facial wrinkles. It efficiently reduces the computation load and the bit cost in transmission. The life-like face is synthesized using a new temporal-spatial blending technique. The feasibility and advantage of the proposed work are demonstrated by video sequences.