A web browser-based multimodal interactive system for netshopping

摘要

This paper introduced the system architecture and design features of a web browser based multimodal prototype system-NetShop. This system is designed to be used for netshopping applications. In this system, we used the context-based interactive query, remedy input and multimodal output to provide a natural interactive environment for this kind of applications. For the multimodal integration, we analyzed the collaborative relations between different modalities and put forward the principal modality driven multimodal integration policy. Furthermore, we developed an open speech programming interface using the plugin and liveconnect technology. It not only helps to enhance the flexibility of the whole system, but also provides an easy way for the third party to develop speech enabled web based applications.