A Novel Document Classification Algorithm Based on Statistical Features and Attention Mechanism

摘要

Bi-directional Long-Short term Memory in Deep Learning is often used to solve the problem of long-term dependency and gradient explosion. The combination of forward and backward sequences also includes more semantic information. However, the influence of keywords in documents is not clearly addressed. Attention mechanism has been successfully used in several start-of-the-art natural language processing related applications. In the field of text processing, the methods of calculating attention weights are typically at the word level. Although these methods improve the performance of a model, they increase the computational cost significantly. In this paper, we propose to calculate attention weights at the structured event level since 1) events contain richer semantics than words or phrases; and 2) event-based attention mechanism reduces computational cost. Different from the existing deep learning model which does not rely on the text statistical features, we add the statistical features on the basis of attention weight calculation. Compared with the existing models, the semantic information contained in the event structure and the corresponding statistical features improves the quality of the text vector representation and achieves better classification performance. Finally, we evaluate the performance of our model in terms of accuracy, recall and F-Score. The experimental results show that our model achieves better results while reducing the computational cost.