tailieunhanh - Báo cáo khoa học: "Adaptive String Distance Measures for Bilingual Dialect Lexicon Induction"

This paper compares different measures of graphemic similarity applied to the task of bilingual lexicon induction between a Swiss German dialect and Standard German. The measures have been adapted to this particular language pair by training stochastic transducers with the ExpectationMaximisation algorithm or by using handmade transduction rules. These adaptive metrics show up to 11% F-measure improvement over a static metric like Levenshtein distance. | Adaptive String Distance Measures for Bilingual Dialect Lexicon Induction Yves Scherrer Language Technology Laboratory LATL University of Geneva 1211 Geneva 4 Switzerland Abstract This paper compares different measures of graphemic similarity applied to the task of bilingual lexicon induction between a Swiss German dialect and Standard German. The measures have been adapted to this particular language pair by training stochastic transducers with the ExpectationMaximisation algorithm or by using handmade transduction rules. These adaptive metrics show up to 11 F-measure improvement over a static metric like Levenshtein distance. 1 Introduction Building lexical resources is a very important step in the development of any natural language processing system. However it is a time-consuming and repetitive task which makes research on automatic induction of lexicons particularly appealing. In this paper we will discuss different ways of finding lexical mappings for a translation lexicon between a Swiss German dialect and Standard German. The choice of this language pair has important consequences on the methodology. On the one hand given the so-ciolinguistic conditions of dialect use diglossia it is difficult to find written data of high quality parallel corpora are virtually non-existent. These data constraints place our work in the context of scarce-resource language processing. On the other hand as the two languages are closely related the lexical relations to be induced are less complex. We argue that this point alleviates the restrictions imposed by the scarcity of the resources. In particular we claim that if two languages are close even if one of them is scarcely documented we can successfully use techniques that require training. Finding lexical mappings amounts to finding word pairs that are maximally similar with respect to a particular definition of similarity. Similarity measures can be based on any level of linguistic analysis .

• Báo cáo khoa học
• Adaptive String Distance Measures
• Bilingual Dialect Lexicon Induction
• Yves Scherrer
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học