ON THE IDENTIFICATION OF MUSICAL NOTES VIA MULTIMODAL INTERACTION

摘要

Learning something new is never easy and musical notes are no exception. Many people find it difficult identifying musical notes when simply hearing different sounds. Therefore, an interface that adds the additional sense of touch to 'feel' notes may increase the user's ability to more accurately identify notes. In this preliminary study, a multimodal interface is developed to teach musical notes. Users can listen, feel, and see the notes using the interface. The interface includes a sound library and a virtual piano that graphically and haptically displays musical notes. In addition, a human factors study is conducted to investigate the effects of haptic feedback on the correct identification of several musical notes. User testing results show that adding haptics to musical notes increases the user's ability to identify the notes that they hear.