tailieunhanh - Báo cáo khoa học: "Minimum Error Rate Training in Statistical Machine Translation"

Often, the training procedure for statistical machine translation models is based on maximum likelihood or related criteria. A general problem of this approach is that there is only a loose relation to the final translation quality on unseen text. In this paper, we analyze various training criteria which directly optimize translation quality. These training criteria make use of recently proposed automatic evaluation metrics. | Minimum Error Rate Training in Statistical Machine Translation Franz Josef Och Information Sciences Institute University of Southern California 4676 Admiralty Way Suite 1001 Marina del Rey CA 90292 och@ Abstract Often the training procedure for statistical machine translation models is based on maximum likelihood or related criteria. A general problem of this approach is that there is only a loose relation to the final translation quality on unseen text. In this paper we analyze various training criteria which directly optimize translation quality. These training criteria make use of recently proposed automatic evaluation metrics. We describe a new algorithm for efficient training an unsmoothed error count. We show that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure. 1 Introduction Many tasks in natural language processing have evaluation criteria that go beyond simply counting the number of wrong decisions the system makes. Some often used criteria are for example F-Measure for parsing mean average precision for ranked retrieval and BLEU or multi-reference word error rate for statistical machine translation. The use of statistical techniques in natural language processing often starts out with the simplifying often implicit assumption that the final scoring is based on simply counting the number of wrong decisions for instance the number of sentences incorrectly translated in machine translation. Hence there is a mismatch between the basic assumptions of the used statistical approach and the final evaluation criterion used to measure success in a task. Ideally we would like to train our model parameters such that the end-to-end performance in some application is optimal. In this paper we investigate methods to efficiently optimize model parameters with respect to machine translation quality as measured by automatic evaluation criteria such as word error .