Attention Window Aware Encoder-Decoder Model for Spoken Language Understanding

摘要

Slot filling task, which aims to predict the semantic slot labels for each specific word in word sequence, is one of the main tasks in Spoken Language Understanding (SLU). In this paper, we propose a variation of encoder-decoder model for sequence labelling. To better use the label dependency feature and prevent overfitting, we use Long Short Term Memory (LSTM) as encoder and Gated Recurrent Unit (GRU) as decoder. We also enhance the model by employing the attention mechanism with attention window as a novel feature, which considers the particularity in slot filling task that each target label corresponds to the specific words and hidden units in the encoder. We test the proposed model using the standard ATIS corpus by adopting different size of attention window. The analysis of trends for the results using different attention window size has shown its application potential of attention window feature.