Category Training Induces Cross-modal Object Representations in the Adult Human Brain

摘要

The formation of cross-modal object representations wasninvestigated using a novel paradigm that was previously successfulnin establishing unimodal visual category learning innmonkeys and humans. The stimulus set consisted of six categoriesnof bird shapes and sounds that were morphed to createndifferent exemplars of each category. Subjects learned newncross-modal bird categories using a one-back task. Over time,nthe subjects became faster and more accurate in categorizingnthe birds. After 3 days of training, subjects were scanned whilenpassively viewing and listening to trained and novel bird types.nStimulus blocks consisted of bird sounds only, bird picturesnonly, matching pictures and sounds (cross-modal congruent),nand mismatching pictures and sounds (cross-modal incongruent).nfMRI data showed unimodal and cross-modal training effectsnin the right fusiform gyrus. In addition, the left STS showedncross-modal training effects in the absence of unimodal trainingneffects. Importantly, for both the right fusiform gyrus and thenleft STS, the newly formed cross-modal representation was specificnfor the trained categories. Learning did not generalize tonincongruent combinations of learned sounds and shapes; theirnresponse did not differ from the response to novel cross-modalnbird types. Moreover, responses were larger for congruent thannfor incongruent cross-modal bird types in the right fusiformngyrus and STS, providing further evidence that categorizationntraining induced the formation of meaningful cross-modal objectnrepresentations. ■