A large part of human communication involves referring to entities in the world and often these entities are objects that are visually present for the interlocutors. A system that aims to resolve such references needs to tackle a complex task: objects and their visual features need to be determined, the referring expressions must be recognised, and extra-linguistic information such as eye gaze or pointing gestures need to be incorporated. Systems that can make use of such information sources exist, but have so far only been tested under very constrained settings, such as WOz interactions. In this paper, we apply to a more complex domain a reference resolution model that works incrementally (i.e., word by word), grounds words with visually present properties of objects (such as shape and size), and can incorporate extra-linguistic information. We find that the model works well compared to previous work on the same data, despite using fewer features. We conclude that the model shows potential for use in a realtime interactive dialogue system.
展开▼
机译：人类通信的大部分涉及在世界上的实体涉及各个实体，并且这些实体通常是视觉上存在的物体。旨在解决这些引用的系统需要解决复杂的任务：需要确定对象和他们的视觉特征，必须识别参考表达，并且需要结合诸如眼睛凝视或指向手势的超语言信息。可以利用此类信息源的系统存在，但到目前为止仅在非常受限的设置下进行测试，例如WOZ交互。在本文中，我们应用于更复杂的域名参考分辨率模型，该参考分辨率模型逐步地工作（即，Word y Word），与视觉上存在对象的属性（例如形状和大小），并且可以包含语言信息。我们发现模型与以前的相同数据相比，效果很好，尽管使用较少的功能。我们得出结论，该模型显示了在实时交互式对话系统中使用的可能性。
展开▼
