首页>
外国专利>
SYSTEM AND METHODS FOR TRAINING TASK-ORIENTED DIALOGUE (TOD) LANGUAGE MODELS
SYSTEM AND METHODS FOR TRAINING TASK-ORIENTED DIALOGUE (TOD) LANGUAGE MODELS
展开▼
机译:面向任务的对话(TOD)语言模型训练系统和方法
展开▼
页面导航
摘要
著录项
相似文献
摘要
Embodiments described herein provide methods and systems for training task-oriented dialogue (TOD) language models. In some embodiments, a TOD language model may receive a TOD dataset including a plurality of dialogues and a model input sequence may be generated from the dialogues using a first token prefixed to each user utterance and a second token prefixed to each system response of the dialogues. In some embodiments, the first token or the second token may be randomly replaced with a mask token to generate a masked training sequence and a masked language modeling (MLM) loss may be computed using the masked training sequence. In some embodiments, the TOD language model may be updated based on the MLM loss.
展开▼