A framework based on (probabilistic) soft logic and neural network for NLP

摘要

Deep neural networks have emerged as a flexible framework that achieved state-of-the-art performance in many NLP applications such as machine translation, named entity recognition, sentiment analysis, and part-of-speech tagging. The main advantage of these neural models is their ability to learn useful representations without hand-engineering features. While this success, these models still suffer from the interpretability issue. More recently, probabilistic soft logic (PSL) is a promising framework based on first-order logic that achieves interesting results in both computer vision and NLP by capturing semantic relationships between entities. Moreover, unifying knowledge-driven modeling approaches and data-driven approaches is a promising framework that will have an exciting impact on structured prediction problems. In this paper, we developed NeuralGLogic a generalization framework of the previous model proposed by Huet al. (2016) that combines deep neural networks with logic rules built either using Soft Logic (SL) or Probabilistic Soft Logic (PSL). Furthermore, we evaluate our framework on different neural network architectures applied to two NLP tasks: sentiment classification and part-of-speech tagging. Experimental results showed that we were able to improve the results over the baselines and outperformed all the previous state-of-the-art systems emphasizing the utility of both SL and PSL rules in reducing the uninterpretability of the neural models thus validating our intuition. (C) 2020 Elsevier B.V. All rights reserved.