Seeing or Not Seeing Your Conversational Partner: The Influence of Interaction Modality on Prosodic Entrainment

摘要

In speech entrainment research, a less investigated yet crucial aspect for modelling multimodal interactive dialogue systems is the influence of interaction modality, i.e. whether conversational partners who are visible to each other would entrain their speech more with respect to when eye contact is inhibited, or not. In our study, we compared prosodic adaptation behaviour (convergence and synchrony) of the same speaker pairs involved in collaborative game sessions under two conditions: audiovisual vs audio-only interaction. Results provide a complex picture, with a tendency to enhance vocal entrainment when the speech/audio channel is the only one available to conversational partners.