Facial features tracker with advanced training for natural rendering of human faces in real-time

摘要

Tracking units for facial features with advanced training for natural rendering of human faces in real-time are provided. An example device receives a video stream, and upon detecting a visual face, selects a 3D model from a comprehensive set of head orientation classes. The device determines modifications to the selected 3D model to describe the face, then projects a 2D model of tracking points of facial features based on the 3D model, and controls, actuates, or animates hardware based on the facial features tracking points. The device can switch among an example comprehensive set of 35 different head orientation classes for each video frame, based on suggestions computed from a previous video frame or from yaw and pitch angles of the visual head orientation. Each class of the comprehensive set is trained separately based on a respective collection of automatically marked images for that head orientation class.