Towards the Realization of Non-verbal Communication - Computer Vision Based Technologies for Analyzing Human Images

摘要

When humans actually meet and directly communicate with each other, not only verbal information, but also non-verbal information plays very important roles. If computers can understand non-verbal information exhibited by humans, more advanced man-machine interface systems could be realized. Non-verbal information used for human-to-human communications include many modalities; in particular, motions of faces and bodies are very important for non-verbal communications. This paper introduces the authors' activities on analyzing human images using computer vision technologies. At ATR Media Integration & Communications Research Laboratories, the authors are aiming at realizing virtual communication environments in which remotely located people can communicate with each other via a virtual scene in a variety of fashions. To realize such a communication environment, the images of real humans need to be analyzed and synthesized in same or different forms in the virtual scene. As an application of the human image analysis and synthesis, the authors have proposed a virtual metamorphosis system in which people can change their forms to any other forms. In the virtual metamorphosis system, facial expressions and body postures of a person need to be estimated in real-time and reproduced in the 3D human model by deforming the model according to the estimated data. More details are described in Section 2. If facial expressions and body gestures can be classified into pre-defined categories by computers, the computers could return preferable responses to the users. The authors are developing algorithms for recognizing facial expressions and hand gestures from camera images as deseribed in Sections 3 and 4, respectively.