Handling Unknown Words in Statistical Machine Translation from a New Perspective

摘要

Unknown words are one of the key factors which drastically impact the translation quality.Traditionally,nearly all the related research work focus on obtaining the translation of the unknown words in different ways.In this paper,we propose a new perspective to handle unknown words in statistical machine translation.Instead of trying great effort to find the translation of unknown words,this paper focuses on determining the semantic function the unknown words serve as in the test sentence and keeping the semantic function unchanged in the translation process.In this way,unknown words will help the phrase reordering and lexical selection of their surrounding words even though they still remain untranslated.In order to determine the semantic function of each unknown word,this paper employs the distributional semantic model and the bidirectional language model.Extensive experiments on Chinese-to-English translation show that our methods can substantially improve the translation quality.