A Multi-Modal Interface for Recognizing Gestures Expressed by Cyclically Repeated Motion of the Hand

摘要

Human naturally uses not only voices but also the gestures composed of cyclically repeated motion of the hand in daily life to express the direction to which he/she wants another person or machine to go. In this paper, we first mention that the vocalization timing correlates closely with the human intention expressed by such gestures. Then we propose a multi-modal method to recognize a gesturer's intention in real-time based on the hand motion velocity at the moment of the gesturer's vocalization estimated by the combination of the optical flow from motion images and the voice trigger extracted from the microphone input. Experimental results of the proposed method applied to mobile robot navigation are also presented.