Interaction between hearing and vision: effects of one modal task on the other modal reaction

摘要

In usual environment, a person recognizes his surroundings using several modalities, especially hearing and vision. Cross modal effect and interaction between the sense of hearing and sight is reported in some researches. In the previous research, no interaction between primal sensory processing of auditory and visual information in brain was reported. However, it was supposed that some interaction may exist in more advanced psychological stage. In the present research, one modal reaction when a subject pays attention to the other modal task is examined to reveal interaction between auditory and visual perception in cognitive stage. Two experiments are carried out. In experiment 1, subjects rate seven kinds of sound environment that are presented to them through headphone when they do visual task that requires them to write down many meaningless characters. In experiment 2, subjects react to visual targets that are presented to them through computer display as soon as possible when they do auditory task that requires them to memorize the contents of stories. These data are compared with normal evaluation of the sound and reaction to the visual target. Results show that auditory impression to several sound sources is changed by doing visual task simultaneously. They also show that normal visual reaction is faster than the reaction under task condition. It is suggested that much attention to one modality causes reduction of attention to the other modality, and that reduced attention leads to variation of impression and decrement of recognition.