Unified Input Processing Framework for Multi-User Multimodal Interaction on Large Display

摘要

This paper proposes the unified input processing framework to enable multimodal, multi-user simultaneous interactions on a large display. This framework defines most common interaction schemes and maps input device's functionality to the common input events in the application. It is designed for a transparent and seamlessly transferrable interface mechanism to enable multiple users to interact with the application in the same ways using various input devices. That is, users can use any input device that mostly suits for them based on the spatial and task constraints. This framework currently supports Kinect-based gesture, a multi-touch panel, and a mobile device. The feasibility of this framework is also discussed through the development of collaborative applications that support multiple user interactions with various input devices.