Toward multimodal notation for mathematics: Why and how

摘要

Notation is a tool of thought, recording and communicating concepts and activities related to a domain of knowledge. In the history of mathematical notation, written notation is usually considered. However, to be an effective thinking tool, notation must be properly perceived. Blind and partially sighted people run into difficulty in making, exploring and understanding mathematical concepts conceived for being represented in multi-dimensional space. In this paper, we capitalize on the multifaceted nature of digital symbols to define multimodal digital notation for graph structures that allows blind and partially sighted people to represent graph structures, reason about them and communicate their reasoning with sighted people as well as each other. As support tool for notation proposed we have designed a multimodal interactive system, in which haptic signals play a crucial role. An evolutionary prototype of the system has been developed and evaluated according to the star life cycle model.