Automatic Evaluation of Translation Quality Using Expanded N-gram Co-occurrence

摘要

BLEU and NIST as official machine translation evaluation metrics are widely used to assess system translation quality. These n-gram co-occurrence algorithms are applied to evaluate language learners' translations in this paper. Subtle differences of evaluation on machine translation and learners' translation are discussed. Dependent on n-gram matching between target translation and references, BLEU and NIST evaluate translation quality completely disregarding the source language. Based on sense overlapping in original language, we make pseudo translations for BLEU and NIST by substituting words or phrases in target translation for synonymous words and phrases in references. Pseudo translations expand n-gram co-occurrence between target translation and references. Evaluation experiments on learners' translation and machine translation corpus with expanded n-gram co-occurrence outperform pure BLEU and NIST evaluation in higher correlation with human assessments.