A multimodal tactile interface for immersive virtual experience

摘要

Multimodal interaction generally allows the user to exploit multiple modes of interfacing with a system. A multimodal interface activates multiple sensory channels to achieve deeper user involvement and enhance the communication with the system. Sensory stimulation is one of the most important factors to reach multimodality and represents a challenge in human-computer interaction to realise enriched virtual representations. Furthermore, playing successfully in a virtual system requires synchronisation among the reproduced signals stimulating the different sensory channels. This paper presents the design and implementation of a multimodal tactile interface to create an immersive virtual experience. The proposed system is the evolution of a previous tactile platform; the main novelties regard the introduction of audio and visual feedbacks to a tactile feedback, signals' synchronisation, and system testing, with the aim to create a robust application to be integrated into novel human-computer interfaces. Tests are carried out to evaluate the degree of synchronism among the three different outputs, system latency, and the quality of multimodal interaction provided based on the ability of users to discriminate different materials by their virtual simulation.