Convolutional neural network with pair-wise pure dependence for sentence classification

摘要

Sentence classification has always been a crucial research topic in Natural Language Processing (NLP). Classical bag-of-words based models have a major limitation: the contextual information between words, which is the key to form meaningful semantic entities, is missing. Moreover, the semantic entities are not necessarily limited to syntactically valid phrases or named entities, but can be high-order association (also referred to as high-order dependence) patterns. To address this issue, in this paper, we propose PPD-CNN, a convolutional neural network (CNN) architecture with Pair-wise Pure Dependence (PPD) for sentence classification. Compared with the traditional CNN, our PPD-CNN (1) combines PPD pattern which is a couple of dependence words as strong un-separable high-level semantic entity and (2) extracts multi-granular semantic information, which treats PPD pattern as an input channel to capture the whole features and the original sentence as another input channel through variable-size convolution filters to catch all kinds of local features. With this design, our PPD-CNN can model the contextual information, which is important for grasping the word sense. The experimental results show that our approach significantly outperforms a wide range of baselines and state-of-the-art methods.