ANALYSIS OF SPEECH AND GESTURES: GESTURE FREQUENCY DURING FLUENT AND HESITANT PHASES IN SPEECH

摘要

This paper describes the processing and analysis of signals pertaining to two communicative modalities, speech and gestures, and examines the nature of their relationship and their temporal correlation. The temporal relationship between speech and hand movements is analyzed using two methods that work at different time scales: in the first method smaller segments called "prosodic phrases" are studied, whereas in the second method larger units are identified based on whether speech within the units is fluent or non-fluent. Both methods of analysis lead to the same conclusion, that speech and gestures are co-expressive and complementary. In this paper we focus on the analysis of gesture frequency during hesitant and fluent phases in speech.