A Knowledge Resources Based Neural Network for Learning Word and Relation Representations

摘要

Using neural networks to train high quality distributed representations of words and multi-relational data has attracted a great attention in recent years. Mapping the words and their relations to low-dimensional continues vector spaces has proved to be useful in natural language processing and information extraction tasks. In this paper, we present a neural network based model that can train word embeddings and relation embeddings taking into account unlabeled text data and knowledge resources jointly. In particular, we use both contexts and definitions of words as neural network inputs to train word embeddings. Based on the word embeddings, we train relation embeddings by defining a proper projecting operation between words. Experiments on various tasks like word similarity and link prediction show that the proposed method can achieve high quality on word and relation representations.