Motion induced pitch: a case of visual-auditory synesthesia

摘要

We report a novel feature-based visual-auditory synesthesia that is more elaborate than the previously reported percepts of synchronized auditory beats induced by visual flashes. CN hears specific pitches and chords when she views different motion patterns including a perfect 5th chord when viewing rotational apparent motion, a tritone chord when the rotation is ambiguous, and complex jarring sounds when viewing dynamic random dots. We discovered that this form of synesthesia builds on simple systematic relationships between visual motion direction and auditory pitch. In the current study, we determined the tuning, the underlying coordinate system, and the sensory impact of this synesthesia. To determine the tuning, we used sinusoidal gratings of 0.12 cycles/deg moving at 22.9 deg/sec, which reliably produced sounds for CN. While viewing a moving sinusoidal grating, CN experiences a 315-Hz sound when the grating moves upward, a 254-Hz sound when it moves horizontally in either direction, and a 217-Hz sound when it moves downward. JNDs were all less than 3-Hz, indicating precise and stable interactions between motion direction and pitch; these relationships were relatively independent of speed. Interestingly, CN experiences no sounds when viewing a square-wave grating, suggesting that the underlying synesthetic interactions are specific to spatial-frequency content as well as direction of motion. We examined the underlying coordinate system of her synesthesia with 90-degree head ti the results indicated that CN's synesthesia arises from a head-centered processing of visual motion. Lastly, to demonstrate the sensory impact of the synesthesia, we used a 2IFC motion-coherence detection task. CN reported using synesthetically induced sounds to detect coherent motion. Indeed, her coherence-detection thresholds were lower than those of non-synesthetic control participants. These results suggest that visual processing of motion direction and auditory perception of pitch can maintain surprisingly specific and systematic neural connections.