Evaluate the Chinese Version of Machine Translation Based on Perplexity Analysis

摘要

Nowadays, the main methods to evaluate the quality of target texts which are translated by machine translation system include BLUE, TER and METEOR. However, based on the frequency of words recurrence, edit distance of translation version and reference version as well as linguistic knowledge, such methods have limitations on deciding the perplexity of Chinese sentences. It is found that grammatical structure of sentence has certain regularity, meanwhile, semantics also has certain collocation rule, so through matching we can judge whether the usage(s) of grammar and semantics accord with the standard usages. Therefore, we figure out the perplexity of Chinese sentences through analyzing the syntax and semantic of Chinese sentences and its main components on the basis of syntactic analysis. According to the experiment, the accuracy of syntax and semantic analysis is 4% higher than BLEU which gets the perplexity by improved weighting target text.