Does sentiment help requirement engineering: exploring sentiments in user comments to discover informative comments

摘要

User comments are valuable resources for software improvement; however, owing to excessive volume, informative comments need to be selectively analyzed. We attempt to address this problem by sentiment analysis and expect sentiment can be a useful indicator for finding informative comments. In this study, we analyze the informative level according to the sentiment of the comment using sentiment analysis. To understand the sentiment in detail, we divide it into four groups and evaluate the characteristics of each group through experiments. Applying topic modeling, we evaluate the informative level of the extracted topic and evaluate the proportion of sentiments by sentiment analysis of the related comments. Additionally, we manually evaluate the informative score of the sample comments in each sentiment group to verify the tendencies observed in the experiments. We find that the probability of being associated with requirements is very low when positive, or when both positive and negative sentiments are weak. In contrast, it has been shown that informative comments are concentrated in negative or strongly negative and positive comments, which are very few among all comments. In particular, the comments observed as strongly positive and negative are highly informative, which is a characteristic that has been overlooked in previous studies. We propose a sentiment model that specifies the sentiment, and confirm sentiments that are highly related to informative comments through sentiment analysis methods and expert evaluations. From these results, it is expected that analyzing negative comments or strongly negative and positive comments can contribute to effective requirement engineering.