A gesture-based method for natural interaction in smart spaces

摘要

A key issue when making spaces smart is the availability of satisfying and personalized interaction methods, for the user to comfortably manage the physical and virtual resources in the environment. Among the multiple interaction approaches that are nowadays being explored with this objective, gesture-based ones seem to have a great potential. In this line, this research describes a gesture-based interaction method that uses a specific grammar to control and network objects in a smart space. The method’s grammar establishes that the user has to identify the object to interact with by performing an individuation gesture primitive (the object’s initial letter), then using an action gesture primitive to indicate the action to perform. The action may involve a second object, which will be identified as the first one. The user will be able to personalize the action gestures in the vocabulary, and to configure the commands assigned to gestures, and the system will provide interaction cues for the user not to feel lost. The main component of the system is a gesture recognition module based on an adapted Dynamic Time Warping algorithm. This module works sharply on acceleration or position data inputs, being suitable to deploy device instrumented or infrastructure-based solutions for gesture recognition (i.e. smartphone or Kinect-based ones). The average recognition rate is 93.63% for smartphone-based recognition and 98.64% for Kinect-based one, respectively. The paper also details the architecture and software tools that enables the interaction method to work in a real environment.