tailieunhanh - Báo cáo khoa học: "Learning to Translate with Multiple Objectives"

We introduce an approach to optimize a machine translation (MT) system on multiple metrics simultaneously. Different metrics (. BLEU, TER) focus on different aspects of translation quality; our multi-objective approach leverages these diverse aspects to improve overall quality. Our approach is based on the theory of Pareto Optimality. | Learning to Translate with Multiple Objectives Kevin Duh Katsuhito Sudoh Xianchao Wu Hajime Tsukada Masaaki Nagata NTT Communication Science Laboratories 2-4 Hikari-dai Seika-cho Kyoto 619-0237 JAPAN kevinduh@ Abstract We introduce an approach to optimize a machine translation MT system on multiple metrics simultaneously. Different metrics . BLEU TER focus on different aspects of translation quality our multi-objective approach leverages these diverse aspects to improve overall quality. Our approach is based on the theory of Pareto Optimality. It is simple to implement on top of existing single-objective optimization methods . MERT PRO and outperforms ad hoc alternatives based on linear-combination of metrics. We also discuss the issue of metric tunability and show that our Pareto approach is more effective in incorporating new metrics from MT evaluation for MT optimization. 1 Introduction Weight optimization is an important step in building machine translation MT systems. Discriminative optimization methods such as MERT Och 2003 Mira Crammer et al. 2006 PRO Hopkins and May 2011 and Downhill-Simplex Nelder and Mead 1965 have been influential in improving MT systems in recent years. These methods are effective because they tune the system to maximize an automatic evaluation metric such as BLEU which serve as surrogate objective for translation quality. However we know that a single metric such as BLEU is not enough. Ideally we want to tune towards an automatic metric that has perfect correlation with human judgments of translation quality. Now at Nara Institute of Science Technology NAIST While many alternatives have been proposed such a perfect evaluation metric remains elusive. As a result many MT evaluation campaigns now report multiple evaluation metrics Callison-Burch et al. 2011 Paul 2010 . Different evaluation metrics focus on different aspects of translation quality. For example while BLEU Papineni et al. .