A Comparative Study of English-Chinese Translations of Court Texts by Machine and Human Translators and the Word2Vec Based Similarity Measure's Ability To Gauge Human Evaluation Biases

摘要

In this comparative study, ajury instruction scenario was used to test the translating capabilities of multiple machine translation tools and a human translator with extensive court experience. Three certified translators/interpreters subjectively evaluated the target texts generated using adequacy and fluency as the evaluation metrics. This subjective evaluation found that the machine generated results had much poorer adequacy and fluency compared with results produced by their human counterpart. Human translators can use strategic omission and explicitation strategies such as addition, paraphrasing, substitution, and repetition to remove ambiguity, and achieve a natural flow in the target language. We also investigate instances where human evaluators have major disagreements and found that human experts could have very biased views. On the other hand, a vord2vec based algorithm, if given a good reference translation, can serve as a robust and reliable similarity reference to quantify human evalutors' biases beacuse it was trained on a large corpus using neural network models. Even though the machine generated versions had better fluency performance compared to their adequacy performance, the human translator's fluency performance was still far superior. The lack of understanding by machine translators led to inaccurate and improper word/phrase selections, which led to bad fluency.