tailieunhanh - Báo cáo khoa học: "Joint Word Segmentation and POS Tagging using a Single Perceptron"

For Chinese POS tagging, word segmentation is a preliminary step. To avoid error propagation and improve segmentation by utilizing POS information, segmentation and tagging can be performed simultaneously. A challenge for this joint approach is the large combined search space, which makes efficient decoding very hard. Recent research has explored the integration of segmentation and POS tagging, by decoding under restricted versions of the full combined search space. | Joint Word Segmentation and POS Tagging using a Single Perceptron Yue Zhang and Stephen Clark Oxford University Computing Laboratory Wolfson Building Parks Road Oxford OX1 3QD UK @ Abstract For Chinese POS tagging word segmentation is a preliminary step. To avoid error propagation and improve segmentation by utilizing POS information segmentation and tagging canbe performed simultaneously. A challenge for this joint approach is the large combined search space which makes efficient decoding very hard. Recent research has explored the integration of segmentation and POS tagging by decoding under restricted versions of the full combined search space. In this paper we propose a joint segmentation and POS tagging model that does not impose any hard constraints on the interaction between word and POS information. Fast decoding is achieved by using a novel multiple-beam search algorithm. The system uses a discriminative statistical model trained using the generalized perceptron algorithm. The joint model gives an error reduction in segmentation accuracy of and an error reduction in tagging accuracy of compared to the traditional pipeline approach. 1 Introduction Since Chinese sentences do not contain explicitly marked word boundaries word segmentation is a necessary step before POS tagging can be performed. Typically a Chinese POS tagger takes segmented inputs which are produced by a separate word seg-mentor. This two-step approach however has an obvious flaw of error propagation since word segmentation errors cannot be corrected by the POS tagger. A better approach would be to utilize POS in formation to improve word segmentation. For example the POS-word pattern number word A a common measure word can help in segmenting the character sequence A A into the word sequence one A measure word A person instead of one A A personal adj . Moreover the comparatively rare POS pattern number word number word can help to prevent .

• Báo cáo khoa học
• Joint Word Segmentation
• POS Tagging
• Single Perceptron
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học