tailieunhanh - Báo cáo khoa học: "Transliteration Alignment"

This paper studies transliteration alignment, its evaluation metrics and applications. We propose a new evaluation metric, alignment entropy, grounded on the information theory, to evaluate the alignment quality without the need for the gold standard reference and compare the metric with F -score. We study the use of phonological features and affinity statistics for transliteration alignment at phoneme and grapheme levels. The experiments show that better alignment consistently leads to more accurate transliteration. In transliteration modeling application, we achieve a mean reciprocal rate (MRR) of on Xinhua personal name corpus, a significant improvement over other reported results on. | Transliteration Alignment Vladimir Pervouchine Haizhou Li Institute for Infocomm Research A STAR Singapore 138632 vpervouchine hli @ Bo Lin School of Computer Engineering NTU Singapore 639798 linbo@ Abstract This paper studies transliteration alignment its evaluation metrics and applications. We propose a new evaluation metric alignment entropy grounded on the information theory to evaluate the alignment quality without the need for the gold standard reference and compare the metric with F-score. We study the use of phonological features and affinity statistics for transliteration alignment at phoneme and grapheme levels. The experiments show that better alignment consistently leads to more accurate transliteration. In transliteration modeling application we achieve a mean reciprocal rate MRR of on Xinhua personal name corpus a significant improvement over other reported results on the same corpus. In transliteration validation application we achieve equal error rate on a large LDC corpus. 1 Introduction Transliteration is a process of rewriting a word from a source language to a target language in a different writing system using the word s phonological equivalent. The word and its transliteration form a transliteration pair. Many efforts have been devoted to two areas of studies where there is a need to establish the correspondence between graphemes or phonemes between a transliteration pair also known as transliteration alignment. One area is the generative transliteration modeling Knight and Graehl 1998 which studies how to convert a word from one language to another using statistical models. Since the models are trained on an aligned parallel corpus the resulting statistical models can only be as good as the alignment of the corpus. Another area is the transliteration validation which studies the ways to validate transliteration pairs. For example Knight and Graehl 1998 use the lexicon frequency Qu and Grefen-stette