What co-speech gestures do:udinvestigating the communicative role ofudvisual behaviour accompanyingudlanguage use during reference inudinteraction

摘要

Language and gesture are thought to be tightly interrelated andudco-expressive behaviours (McNeill, 1992; 2005) that, when used inudcommunication, are often referred to as composite signals/utterancesud(Clark, 1996; Enfield, 2009). Linguistic research has typicallyudfocussed on the structure of language, largely ignoring theudeffect gesture can have on the production and comprehension ofudutterances. In the linguistic literature, gesture is shoehorned intoudthe communicative process rather than being an integral part ofudit (Wilson and Wharton, 2006; Wharton, 2009), which is at oddsudwith the fact that gesture regularly plays a role that is directlyudconnected to the semantic content of, in Gricean terms, “what isudsaid” (Kendon, 2004; Grice, 1989). In order to explore these issues,udthis thesis investigates the effect of manual gestures on interactionudat several different points during production and comprehension,udbased on the Clarkian Action Ladder (Clark, 1996). It focusses onudthe top two levels of the ladder: Level 3 signaling and recognisingudand level 4 proposing and considering. In doing so, it exploresudgesture’s local effect on how utterances are composed and comprehended,udbut also its more global effect on the interactional structureudand the goals of the participants. This is achieved throughudtwo experiments. The first experiment, the map task, is an interactiveudspatial description task and the second is an eye-trackedudvisual world task. These two experiments explore how gesturesudare composed during the map task, how gestures affect the realtimeudcomprehension of utterances, and how gestures are embeddedudwithin the turn-by-turn nature of talk. This thesis builds audpicture of the effect of gesture at each stage of the comprehensionudprocess, demonstrating that gesture needs to be incorporated fullyudinto pragmatic models of communication.