Incorporating Background Knowledge into Dialogue Generation Using Multi-task Transformer Learning

摘要

Knowledge plays a very important role in the dialogue systems. Inspired by how humans use unstructured background knowledge in the conversations, this paper proposes a dialogue generation model based on multi-task learning. The model divides the conversation generation task into two tasks, a knowledge selection task and a response prediction task, which are regard as a reading comprehension task and a text generation task separately. Specifically, in the task of knowledge selection, a language pre-training model Bidirectional Encoder Representations from Transformers (BERT) is applied to solve the problem of selecting the knowledge from the background knowledge documents in the current context. And in the task of response prediction, a transformer version of pointer-generator network, being composed of an encoder using the shared BERT mentioned in knowledge selection and a decoder using the left-context-only transformer, is applied to copy tokens from the background knowledge via pointing and produce tokens in the vocabulary through a generator. Our experiments on the HOLL-E dataset show that our model achieves better results than the strong baseline models and the related recent work.