Facial video coding/decoding at ultra-low bit-rate: a 2D/3D model-based approach

摘要

A real-time facial video coding/decoding system was proposed based on the establishment of 2D/3D mixed coding/decoding scheme. It has better rate/distortion performance at ultra-low bit-rate. Multi-measurements and online appearance models were applied to track the 3D facial motion from video by the improved particle filtering. 3D facial animation was produced by combining the parameterized model and muscular model. 3D hair was synthesized based on the hair detection result in video. 3D coding/decoding result of foreground and 2D coding/decoding result of background were stitched seamlessly. At ultra-low bit-rate, the objective experiment confirmed the comprehensive advantage between coding efficiency and decoding quality of this system, and the subjective experiment indicated the suitability of subjective face identification by it.