tailieunhanh - Báo cáo khoa học: "Automatic Evaluation Method for Machine Translation using Noun-Phrase Chunking"

As described in this paper, we propose a new automatic evaluation method for machine translation using noun-phrase chunking. Our method correctly determines the matching words between two sentences using corresponding noun phrases. Moreover, our method determines the similarity between two sentences in terms of the noun-phrase order of appearance. Evaluation experiments were conducted to calculate the correlation among human judgments, along with the scores produced using automatic evaluation methods for MT outputs obtained from the 12 machine translation systems in NTCIR7. Experimental results show that our method obtained the highest correlations among the methods in both sentence-level adequacy and. | Automatic Evaluation Method for Machine Translation using Noun-Phrase Chunking Hiroshi Echizen-ya Hokkai-Gakuen University S 26-Jo W 11-chome Chuo-ku Sapporo 064-0926 Japan echi@ Abstract As described in this paper we propose a new automatic evaluation method for machine translation using noun-phrase chunking. Our method correctly determines the matching words between two sentences using corresponding noun phrases. Moreover our method determines the similarity between two sentences in terms of the noun-phrase order of appearance. Evaluation experiments were conducted to calculate the correlation among human judgments along with the scores produced using automatic evaluation methods for MT outputs obtained from the 12 machine translation systems in NTCIR-7. Experimental results show that our method obtained the highest correlations among the methods in both sentence-level adequacy and fluency. 1 Introduction High-quality automatic evaluation has become increasingly important as various machine translation systems have developed. The scores of some automatic evaluation methods can obtain high correlation with human judgment in document-level automatic evalua-tion Coughlin 2007 . However sentence-level automatic evaluation is insufficient. A great gap exists between language processing of automatic evaluation and the processing by humans. Therefore in recent years various automatic evaluation methods particularly addressing sentence-level automatic evaluations have been proposed. Methods based on word strings . BLEU Papineni et al. 2002 NIST NIsT 2002 METEOR Banerjee and Lavie. 2005 ROUoE-L Lin and Och 2004 Kenji Araki Hokkaido University N 14-Jo W 9-Chome Kita-ku Sapporo 060-0814 Japan araki@ and IMPACT Echizen-ya and Araki 2007 calculate matching scores using only common words between MT outputs and references from bilingual humans. However these methods cannot determine the correct word correspondences sufficiently .