tailieunhanh - Báo cáo khoa học: "Maximum Entropy Based Restoration of Arabic Diacritics"

Short vowels and other diacritics are not part of written Arabic scripts. Exceptions are made for important political and religious texts and in scripts for beginning students of Arabic. Script without diacritics have considerable ambiguity because many words with different diacritic patterns appear identical in a diacritic-less setting. We propose in this paper a maximum entropy approach for restoring diacritics in a document. The approach can easily integrate and make effective use of diverse types of information; the model we propose integrates a wide array of lexical, segmentbased and part-of-speech tag features. . | Maximum Entropy Based Restoration of Arabic Diacritics Imed Zitouni Jeffrey S. Sorensen Ruhi Sarikaya IBM . Watson Research Center 1101 Kitchawan Rd Yorktown Heights NY 10598 izitouni sorenj sarikaya @ Abstract Short vowels and other diacritics are not part of written Arabic scripts. Exceptions are made for important political and religious texts and in scripts for beginning students of Arabic. Script without diacritics have considerable ambiguity because many words with different diacritic patterns appear identical in a diacritic-less setting. We propose in this paper a maximum entropy approach for restoring diacritics in a document. The approach can easily integrate and make effective use of diverse types of information the model we propose integrates a wide array of lexical segmentbased and part-of-speech tag features. The combination of these feature types leads to a state-of-the-art diacritization model. Using a publicly available corpus LDC s Arabic Treebank Part 3 we achieve a diacritic error rate of a segment error rate and a word error rate of . In case-ending-less setting we obtain a diacritic error rate of a segment error rate and a word error rate of . 1 Introduction Modern Arabic written texts are composed of scripts without short vowels and other diacritic marks. This often leads to considerable ambiguity since several words that have different diacritic patterns may appear identical in a diacritic-less setting. Educated modern Arabic speakers are able to accurately restore diacritics in a document. This is based on the context and their knowledge of the grammar and the lexicon of Arabic. However a text without diacritics becomes a source of confusion for beginning readers and people with learning disabilities. A text without diacritics is also problematic for applications such as text-to-speech or speech-to-text where the lack of diacritics adds another layer of ambiguity when processing the data. As an example .

• Báo cáo khoa học
• Maximum Entropy
• Based Restoration
• Arabic Diacritics
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Income inequality
• Optimal taxation
• Maximum entropy algorithm
• Alternative maximum entropy approach
• Optimal taxation policy
• Shannon entropy
• MB distribution
• Maximum entropy approach
• Variation of political temperature
• Intense publicity
• luận văn
• công nghệ thông tin
• ngăn chặn wed xấu
• mô hình cực đại entropy
• nội dung web
• mô hình maximum entropy
• Portfolio optimization
• Maxi mum entropy Approach
• Financial instruments
• Heuristic maximum entropy approach
• Non negative
• Domain Adaptation
• Language Models
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• Iterative Scaling
• Coordinate Descent Methods
• Language Dynamics and Capitalization
• Fernando Batistaa
• Maximum Entropy Based Phrase Reordering Model
• Statistical Machine Translation
• Deyi Xiong Qun Liu and Shouxun Lin
• Combining Lexical
• Syntactic
• Semantic Features
• Maximum Entropy Models
• Extracting Relations
• Improvement of a Whole
• Sentence Maximum Entropy Language Model
• Grammatical Features
• Refined Lexicon Models
• Feature Lattices
• Maximum Entropy Modelling
• Andrei Mikheev
• Analysis System of Speech Acts
• Discourse Structures
• Maximum Entropy Model
• Maximum Entropy Mode
• l Learning of the Translation Rules
• Kengo Sato and Masakazu Nakanishi
• Semi Supervised
• Maximum Entropy Based Approach to Acronym
• Abbreviation Normalization in Medical Texts
• Discriminative Training
• bài toán phân lớp quan điểm
• thiết kế hệ thống
• lập trình hệ thống
• xây dựng phần mềm
• kỹ thuật lập trình
• Japanese Dependency Structure Analysis
• Kiyotaka Uchimoto
• scientific reports
• model language
• process natural language
• Comparison of Alignment Templates
• Natural Language Understanding
• Investigating GIS and Smoothing
• Maximum Entropy Taggers
• James R
• Curran
• Predictive modelling
• Logistic regression
• Ecological niche
• Plant habitats
• Luận văn Thạc sỹ Kỹ thuật phần mềm
• Phân lớp quan điểm
• Hệ thống VNU SMM
• Ứng dụng mô hình Maximum Entropy
• Luận văn thậc sĩ Kỹ thuật phần mềm
• Luận văn thậc sĩ
• Kỹ thuật phần mềm
• Dữ liệu văn bản