Investigating text-picture integration in videos with the multimedia contradiction paradigm

摘要

In two experiments, the multimedia contradiction paradigm was used to investigate whether learners map information conveyed through the audio and the picture track of a video. In Experiment 1 (N = 85), the information conveyed through the audio track and the picture track was always consistent (control group) or was made inconsistent by changing the audio track at one point in time (text-wrong group). Experiment 2 (N = 143) added a second inconsistent condition by changing the picture track at one point in time (picture-wrong group). In both experiments, the learners' gaze behaviour differed from that of the control group when inconsistent information was presented in the experimental groups. This indicates that mapping processes-which are an indispensable part of integration-occur when learners process videos. Regarding learning outcomes, no differences between groups were observed. In addition, only a few learners remembered the conflict after learning. Further, recall shifted towards the pictorial information when learners encountered conflict.