Tracking a User's Face and Hands in Three Dimensions in Real Time

摘要

This work outlines a system in which a stereo camera may effectively track a user's face and hands in three dimensions. Given this information, a method for controlling objects in three dimensions is also described. The system begins by finding faces. If more than one face is found in the image, the algorithm uses depth information to isolate the face that is closest to the camera. The algorithm then gathers information about the user's skin tone by examining the content of the face found. For much of the processing, only the hue and saturation components are used after applying an HSV to RGB transformation given the camera output. The skin tone information in tandem with depth is then used to isolate the user's hands, and track them in three dimensions. To be used as an effective interface, the system uses information of the two hands relative to the user's face. In controlling an object in three dimensions, if the user would like to move the object up, he or she simply positions both hands above his or her face. Similar commands allow the user to apply a translational factor in three dimensions, as well as applying yaw and roll when wanted.