Recurrent neural network language model (RNNLM) has been proved to be more competitive than other neural network language models. However, the input-layer of most current RNNLMs only uses one single feature i.e. the index of the word, which is a unit vector. Previous studies proved that language models with additional linguistic information achieve better performance. In this study, the vector space word representations (word vector), which can capture syntactic and semantic regularities of language, is used as additional features in order to enhance RNNLM. Finally, experimental results showed that the word vector features is very useful to improve the performance of RNNLM. Evaluated on a Mandarin test set, 10% relative reduction on perplexity could be obtained and 0.5 points absolute character error rate reductions could be obtained, compared to the conventional RNNLM.
展开▼
