ON THE IDENTIFICATION OF MUSICAL NOTES VIA MULTIMODAL INTERACTION

摘要

Learning something new is never easy and musical notesrnare no exception. Many people find it difficult identifyingrnmusical notes when simply hearing different sounds.rnTherefore, an interface that adds the additional sense ofrntouch to ‘feel’ notes may increase the user’s ability to morernaccurately identify notes. In this preliminary study, arnmultimodal interface is developed to teach musical notes.rnUsers can listen, feel, and see the notes using the interface.rnThe interface includes a sound library and a virtual pianornthat graphically and haptically displays musical notes. Inrnaddition, a human factors study is conducted to investigaternthe effects of haptic feedback on the correct identificationrnof several musical notes. User testing results show thatrnadding haptics to musical notes increases the user’s abilityrnto identify the notes that they hear.