COMET - Deploying a New State-of-the-art MT Evaluation Metric in Production

摘要

Metrics in a commercial setting: 1. Automatic metrics like BLEU are of limited use 2. Adaptive evaluation frameworks trained to correlate well provide an attractive solution 3. Our COMET framework is publicly available Evaluating Metrics: 1. Metrics can have different use cases and applications 2. A tiered evaluation method can help to align expectations 3. Considering the statistical significance of modelling decisions can be insightful.