Task-Oriented Synergistic Multimodality

摘要

Multimodal interface aims at overcoming the shortcomings of conventional human-computer communication -- serialized input, excessive accuracy, and low input efficiency -- by exploiting the complementary nature of multiple modalities in interpreting user intent, thus improving the efficiency and naturalness of human-computer interaction. In this paper, we present motivations driving multimodal interface research and give a brief introduction to the ideas underlying our MUIR~* project. We put forward a task-oriented framework of multimodal interface research, which consists of our concept of multimodality and perspectives concerning sufficiency vs. accuracy, multimodal integration approach, and an agent-based architecture for multimodal interface.