tailieunhanh - Báo cáo khoa học: "Transliteration Alignment"

This paper studies transliteration alignment, its evaluation metrics and applications. We propose a new evaluation metric, alignment entropy, grounded on the information theory, to evaluate the alignment quality without the need for the gold standard reference and compare the metric with F -score. We study the use of phonological features and affinity statistics for transliteration alignment at phoneme and grapheme levels. The experiments show that better alignment consistently leads to more accurate transliteration. In transliteration modeling application, we achieve a mean reciprocal rate (MRR) of on Xinhua personal name corpus, a significant improvement over other reported results on. | Transliteration Alignment Vladimir Pervouchine Haizhou Li Institute for Infocomm Research A STAR Singapore 138632 vpervouchine hli @ Bo Lin School of Computer Engineering NTU Singapore 639798 linbo@ Abstract This paper studies transliteration alignment its evaluation metrics and applications. We propose a new evaluation metric alignment entropy grounded on the information theory to evaluate the alignment quality without the need for the gold standard reference and compare the metric with F-score. We study the use of phonological features and affinity statistics for transliteration alignment at phoneme and grapheme levels. The experiments show that better alignment consistently leads to more accurate transliteration. In transliteration modeling application we achieve a mean reciprocal rate MRR of on Xinhua personal name corpus a significant improvement over other reported results on the same corpus. In transliteration validation application we achieve equal error rate on a large LDC corpus. 1 Introduction Transliteration is a process of rewriting a word from a source language to a target language in a different writing system using the word s phonological equivalent. The word and its transliteration form a transliteration pair. Many efforts have been devoted to two areas of studies where there is a need to establish the correspondence between graphemes or phonemes between a transliteration pair also known as transliteration alignment. One area is the generative transliteration modeling Knight and Graehl 1998 which studies how to convert a word from one language to another using statistical models. Since the models are trained on an aligned parallel corpus the resulting statistical models can only be as good as the alignment of the corpus. Another area is the transliteration validation which studies the ways to validate transliteration pairs. For example Knight and Graehl 1998 use the lexicon frequency Qu and Grefen-stette

• Báo cáo khoa học
• Transliteration Alignment
• Vladimir Pervouchine
• Haizhou Li
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• An Algorithm
• Unsupervised Transliteration Mining
• Application to Word Alignment
• ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học