Enhancing Recurrent Neural Networks with Positional Attention for Question Answering

摘要

Attention based recurrent neural networks (RNN) have shownrna great success for question answering (QA) in recent years. Althoughrnsignificant improvements have been achieved over the nonattentivernmodels, the position information is not well studied withinrnthe attention-based framework. Motivated by the effectiveness ofrnusing the word positional context to enhance information retrieval,rnwe assume that if a word in the question (i.e., question word) occursrnin an answer sentence, the neighboring words should be givenrnmore attention since they intuitively contain more valuable informationrnfor question answering than those far away. Based on thisrnassumption, we propose a positional attention based RNN model,rnwhich incorporates the positional context of the question wordsrninto the answers’ attentive representations. Experiments on twornbenchmark datasets show the great advantages of our proposedrnmodel. Specifically, we achieve a maximum improvement of 8.83%rnover the classical attention based RNN model in terms of meanrnaverage precision. Furthermore, our model is comparable to if notrnbetter than the state-of-the-art approaches for question answering.