基于弱监督预训练深度模型的微博情感分析

摘要

微博情感分析对于商业事务和政治选举等应用非常重要.传统的做法主要基于浅层机器学习模型,对人工提取的特征有较大的依赖,而微博情感特征往往难以提取.深度学习可以自动学习层次化的特征,并被用于解决情感分析问题.随着新的深度学习技术的提出,人们发现只要提供足够多的监督数据,就能训练出好的深度模型.然而,在微博情感分析中,通常监督数据都非常少.微博中广泛存在着弱监督数据.该文提出基于弱监督数据的“预训练—微调整”训练框架(distant pretrain-finetune),使用弱监督数据对深度模型进行预训练,然后使用监督数据进行微调整.这种做法的好处是可以利用弱监督数据学习到一个初始的模型,然后利用监督数据来进一步改善模型并克服弱监督数据存在的一些问题.我们在新浪微博数据上进行的实验表明,这种做法可以在监督数据较少的情况下使用深度学习,并取得比浅层模型更好的效果.%Sentiment analysis (SA) is important in many applications such as commercial business and political election.The state-of-the-art methods of SA are based on shallow machine learning models.These methods are heavily dependent on feature engineering,however,the features for Weibo SA are difficult to be extracted manually.Deep learning (DL) can learn hierarchical representations from raw data automatically and has been applied for SA.Recently proposed DL techniques shown that one can train deep models successfully given enough supervised data.However,in Weibo SA,supervised data are usually too scarce.It is easy to obtain large scale distant supervision data in Weibo.In this paper,we proposed to pre-train deep models by distant supervision and used supervised data to fine-tune the deep models.This approach could take the advantages of distant supervision to learn good initial models while using supervised data to improve the models and to correct the errors brought by distant supervision.Experimental results on Sina Weibo dataset show that we can train deep models with small scale supervised data and obtain better results than shallow models.