Recent research has demonstrated that goal-oriented dialogue agents trained on large datasels can achieve striking performance when interacting with human users. In real world applications, however, it is important to ensure that the agent performs smoothly interacting with not only regular users but also those malicious ones who would attack the system through interactions in order to achieve goals for their own advantage. In this paper, we develop algorithms to evaluate the robustness of a dialogue agent by carefully designed attacks using adversarial agents. Those attacks are performed in both black-box and white-box settings. Furthermore, we demonstrate that adversarial training using our attacks can significantly improve the robustness of a goal-oriented dialogue system. On a case-study of the negotiation agent developed by (Lewis et al., 2017), our attacks reduced the average advantage of rewards between the attacker and the trained RL-based agent from 2.68 to -5.76 on a scale from -10 to 10 for randomized goals. Moreover, with the proposed adversarial training, we are able to improve the robustness of negotiation agents by 1.5 points on average against all our attacks.
展开▼
机译：最近的研究表明，在大型数据集上进行培训的面向目标的对话代理在与人类用户进行交互时可以达到惊人的性能。但是，在现实世界的应用程序中，重要的是要确保代理不仅与常规用户而且与那些会通过交互攻击系统以实现自身优势的恶意用户进行顺畅的交互。在本文中，我们开发了算法，通过使用对抗性代理精心设计的攻击来评估对话代理的鲁棒性。这些攻击是在黑盒和白盒设置中进行的。此外，我们证明了使用我们的攻击进行对抗训练可以显着提高面向目标的对话系统的鲁棒性。在（Lewis et al。，2017）开发的协商代理的案例研究中，我们的攻击将攻击者和经过培训的基于RL的代理之间的奖励平均优势从2.68降低到-5.76，等级从-10降低到-5.76。随机目标为10。此外，通过提议的对抗训练，我们能够针对所有攻击将协商代理的鲁棒性平均提高1.5分。
展开▼
