TransPhrase: A new method for generating phrase embedding from word embedding in Chinese

摘要

Currently, there are two main methods of learning phrase embedding: the distribution method and the composition method. The distribution method treats a phrase as an entirety and learns phrase embedding based on the context of the phrase. Its disadvantage is that it completely ignores the semantics of the component words of the phrase and the data sparseness problem. The composition method calculates the phrase embedding from the embedding of component words. The existing composition methods fail to represent the semantics of phrases well. Because of the above problems, we take Chinese, for example, and propose a new composition method to generate phrase embedding from the component word embedding, named TransPhrase. It is a neural network that can use LSTM to learn the order information of component words, use the attention mechanism to learn the important information of component words, and use a fully connected network to learn the semantic information of component words, and finally predict phrase embedding. It can solve the data sparseness problem and properly and fully represent the semantics of phrases. Our evaluation of three Chinese phrase-level semantic tasks shows that the comprehensive performance of TransPhrase's phrase representation is better than the composition method, the distribution method, and the pre-trained language model.