Study on text representation method based on deep learning and topic information

摘要

Deep learning provides a new modeling method for natural language processing. In recent years, it has been applied in language model, text classification, machine translation, sentiment analysis, question and answer system, word distributed representation, etc., and a series of theoretical research results have been obtained. For the text representation task, this paper studies the strategy of fusing global and local context information, and proposes a word representation model called Topic-based CBOW that integrates deep neural network, topic information and word order information. Then, based on the word distributed representation obtained by Topic-based CBOW, a short text representation method with TF-IWF-weighted pooling is proposed. Finally, the performance of the Topic-based CBOW model and the short text representation are compared with the baseline models, and it is found that the proposed method improves the quality of the word distributed representation to some extent by introducing the topic vector and retaining word order information, and text representation also performs well in text classification tasks.