深度置信网络的Spark并行化在微博情感分类中的应用研究

摘要

Chinese micro blog sentiment analysis can be found that the public's attitude toward hot events and grasp the network public opinion,thus become a hot research in the text mining.This paper put forwards the parallelization of deep belief networks for Chinese micro blog sentiment analysis by Spark.Firstly,the Word2Vec tool was used to express the microblogging text and the establishment of the emotional dictionary.Then,the microblogging emotion classification model was constructed by using the deep confidence network.Finally,the neural network of the deep confidence neural network was processed by the Spark cluster.The experimental results showed that the microblogging emotion classification model based on deep confidence network was parallelized under the Spark platform, and the training time was shortened,and the accuracy of emotion classification was 5% higher than that of the traditional shallow learning method.%中文微博情感分析可以发现公众对热点事件的态度掌握网络舆情,因此成为文本挖掘的一个热点研究.采用一种基于Spark并行化的深度置信网络的情感分类方法,该方法利用Word2Vec工具表示微博文本和建立情感词典;使用深度置信网络构建微博情感分类模型;通过Spark集群对深度置信神经网络进行并行化处理.实验结果表明,基于深度置信网络的微博情感分类模型在Spark平台下并行化,训练时间大幅缩短,情感分类的准确率比传统的浅层学习方法高5%.