Persian Language Modeling Using Recurrent Neural Networks

摘要

In this paper, recurrent neural networks are applied to language modeling of Persian, using word embedding as word representation. To this aim, unidirectional and bidirectional Long Short-Term Memory (LSTM) networks are used, and the perplexity of Persian language models on a 100-million-word data set is evaluated. The effect of various parameters, including number of hidden layers and size of LSTM units, on the performance of the networks in reducing the perplexity of the models are investigated. Among different LSTM language models, the best perplexity, which is equal to 59.05, is achieved from a 2-layer bidirectional LSTM model. Comparing this value with the perplexity of the classical Trigram model, which is equal to 138, an improvement in the modeling is noticeable, which is due to the ability of neural networks to make a higher generalization in comparison with the well-known N-gram model.