Multi-modal gesture based interactive system and method using one single sensing system

摘要

Described herein is a method and a system for providing efficient and complementary natural multi-modal gesture based interaction with a computerized system which displays visual feedback information on a graphical user interface on an interaction surface. The interaction surface is within the frustum of an imaging device comprising a single sensing system. The system uses the single sensing system for detecting both touch gesture interactions with the interaction surface (120) and three-dimensional touch-less gesture interactions in areas or volumes above the interaction surface performed by hands of a user. Both types of interaction are associated contextually with an interaction command controlling the computerized system when the gesture has been detected. The system comprises preferably a projection system for displaying the graphical user interface and visual feedback on the interaction surface, the projection system being locatable on the same side or on the opposite side of the interaction surface to the sensing system.