Embeddings and Convolution, Is That the Best You can Do with Sentiment Features?

摘要

Rapid growth of digital media motivates research on machine-assisted text analysis. Sentiment analysis, among one of the prevalent applications, has drawn great attention. In addition to the traditional bag-of-words models, embedding methods have become de facto standard for text representation, and various convolutional, recurrent and recursive neural networks are dominating leaderboards. Despite the large number of deep learning models in publication, the performance benchmarks in sentiment analysis are approaching a limit. If language-specific syntactic and semantic knowledge is excluded, is there still room for significant improvements? Over a general neural network that is based on word embedding, 2D convolution and max-pooling, we conduct extensive experiments on its various components, including convolutional kernels, pooling methods, recurrent layers, and attention mechanism. Certain combinations show moderate improvements in classification accuracy which are comparable to more sophisticated networks, but no sign of major breakthrough is in sight. We also extend the scope with potential game changers, covering context-aware representations, linguistic information, and large scale knowledge transfer in natural languages. Reported metrics show their great value in breaking the current performance bottleneck.