A USER DRIVEN EXPERIENCE IN THE DESIGN OF A MULTIMODAL INTERFACE FOR INDUSTRIAL DESIGN REVIEW

摘要

In this work we present a novel approach to author multimodal dialogues through a graph-based user-centric approach tailored to virtual design reviews. Distinct modalities and the interaction dialogue itself need to be highly configurable according to each user's specific preferences and specific scenario. We will show how modalities can be bound together via a bidirectional graph in an authoring tool to allow the specification of application-specific domain keywords and user-centered composition of modalities. Modalities thus shall not be hard-wired to the application itself. Instead, we provide a complete persistent definition of all used modalities outside the application, including gestures, voice and the graphical user interface. Through incremental navigation within the graph, the user is enabled to specify precise modalities for interactions. Specifically, nodes and edges of the graph represent the dialogue of the user with the system. Furthermore, they reflect the application scenario through their names. Using such a flexible approach to interaction configurability has required to resolve the application functionalities in a flexible manner because of the complete separation of user interaction and application response.