Visual information about speech content from the talker’s mouth is often available before auditory information from the talker's voice. Here we examined perceptual and neural responses to words with and without this visual head start. For both types of words, perception was enhanced by viewing the talker's face, but the enhancement was significantly greater for words with a head start. Neural responses were measured from electrodes implanted over auditory association cortex in the posterior superior temporal gyrus (pSTG) of epileptic patients. The presence of visual speech suppressed responses to auditory speech, more so for words with a visual head start. We suggest that the head start inhibits representations of incompatible auditory phonemes, increasing perceptual accuracy and decreasing total neural responses. Together with previous work showing visual cortex modulation (Ozker et al., 2018b) these results from pSTG demonstrate that multisensory interactions are a powerful modulator of activity throughout the speech perception network.
展开▼
机译：通常会先获得来自讲话者口中语音内容的视觉信息，然后再获得来自讲话者语音中的听觉信息。在这里，我们研究了在有和没有这种视觉先启的情况下对单词的知觉和神经反应。对于这两种类型的单词，通过查看说话者的脸部都可以增强感知能力，但是对于具有起始功能的单词而言，增强作用明显更大。从癫痫患者后颞上回（pSTG）的听觉联想皮层上植入的电极测量神经反应。视觉语音的存在抑制了对听觉语音的响应，对于具有视觉起步的单词更是如此。我们建议，从头开始会抑制听觉音素不兼容的表示，从而增加感知准确度并降低总神经反应。结合先前显示视觉皮质调制的工作（Ozker et al。，2018b），来自pSTG的这些结果表明，多感觉相互作用是整个语音感知网络中强大的活动调节剂。
展开▼
