Ideas on Multi-layer Dialogue Management for Multi-party, Multi- conversation, Multi-modal Communication (Extended Abstract of Invited Talk)

摘要

Most current dialogue systems concern only a short dialogue between a single system and single user, focused on a single task. On the other hand, the full spectrum of communication between interacting agents includes cases in which multiple segments of conversation can be interleaved with other, sometimes unrelated actions and events (e.g., a cocktail party). Language use in the Mission Rehearsal Exercise Project at ICT (Swartout, Hill, Gratch, Johnson, Kyriakakis, Labore, Lindheim, Marsella, Miraglia, Moore, Morie, Rickel, Thiebaux, Tuch, Whitney and Douglas 2001) falls between these two extremes, having one main purpose (Army platoon-level leadership training using virtual reality and virtual humans), but multiple characters, each with its own goals, interests, and capabilities. In this scenario, multiple characters must engage in dialogue, both with each other and with the human trainee. Moreover, multiple conversations are involved, each with a distinct context for interpretation. The conversations are also multimodal in two senses. First, communication can occur not just with speech, but also with visual media including gesture and gaze of artificial characters, and secondly different media sets must be used for different communications, e.g., face-to-face communication for some characters and radio communication for others who are not physically co-present. We present here a multiple layer approach towards modelling and managing these complexities, including who is accessible for conversation, paying attention, involved in a conversation, as well as turn- taking, initiative, grounding, and higher-level dialogue functions. The method will follow that used in the Trindi project, where one specifies an information state, and 'dialogue moves' representing input and output, as well as associated updates to information state.