Topic detection using BNgram method and sentiment analysis on twitter dataset

摘要

Online social and news media has become a very popular for users to share their opinions. It generates rich and timely information about actual world actions of all types. Several efforts were dedicated for mining topics, sentiments and opinions automatically from natural language in news, social media messages, and commercial reviews of product and services. Social media like facebook, twitter, online review websites like Amazon are popular sites where millions of users exchange their opinions and making it a valuable platform for tracking and analyzing trending topics and sentiments. This provides important information for decision making in various domains. An enormous amount of available data requires information filtering for drilling down the relevant topics and events. Topic detection is the solution for monitoring and summarizing information generates from social sources. Various topic detection methods are available which affect the quality of result. In this paper we use BNgram which is one of the novel topic detection methods on three large Twitter datasets associated to recent events. It has been observed that the pre-processing of the data and sampling procedure are greatly affecting the quality of detected topics. On much focused topics, standard NLP techniques can do well for social streams. But for handling more heterogeneous streams novel techniques are used. BNgram method gives the best performance, thus being more reliable. In this paper we also find the sentiments of people related to events. “Sentiwordnet dictionary” is used for finding scores of each word. And then sentiments are classified as “negative, positive and neutral”.