The importance of gaze and gesture in interactive multimodal explanation

摘要

The objective of this research is twofold. Firstly, we argue that gaze and gesture play an essential part in interactive explanation and that it is thus a multi-modal phenomenon. Two corpora are analyzed: (1) a group of teacher novices and experts and (2) a student teacher dyad, both of whom construct explanations of students' reasoning after viewing videos of student dyads who are solving physics problems. We illustrate roles of gaze in explanations constructed within a group and roles of gesture in explanation constructed within a dyad. Secondly, we show how the analysis of such knowledge-rich empirical data pinpoints particular difficulties in designing human-computer interfaces that can support explanation between humans, or a fortiori, that can support explanation between a human and a computer.