Multimodal Interaction for Wearable Augmented Reality Environments

摘要

We describe an approach to natural 3D multimodal interaction in immersive environments. Our approach fuses symbolic and statistical information from a set of 3D gesture and speech agents, building in part on prior research on disambiguating the user's intent in 2D and 2.5D user interfaces. We present an experimental system architecture that embodies this approach, and provide examples from a preliminary 3D multimodal testbed to explore our ideas in augmented and virtual reality.