tailieunhanh - Báo cáo khoa học: "Japanese Morphological Analyzer using Word "

We developed a Japanese morphological analyzer that uses the co-occurrence of words to select the correct sequence of words in an unsegmented Japanese sentence. The co-occurrence information can be obtained from cases where the system incorrectly analyzes sentences. As the amount of information increases, the accuracy of the system increases with a small risk of degradation. Experimental results show that the proposed system assigns the correct phonological representations to unsegmented Japanese sentences more precisely than do other popular systems. . | Japanese Morphological Analyzer using Word Co-occurrence JTAG Takeshi FUCHI NTT Information and Communication Systems Laboratories Hikari-no-oka 1-1 Yokosuka 239-0847 Japan fuchi@ Shinichữo TAKAGI NTT Information and Communication Systems Laboratories Hikari-no-oka 1-1 Yokosuka 239-0847 Japan takagi@ Abstract We developed a Japanese morphological analyzer that uses the co-occurrence of words to select the correct sequence of words in an unsegmented Japanese sentence. The co-occurrence information can be obtained from cases where the system incoưectly analyzes sentences. As the amount of information increases the accuracy of the system increases with a small risk of degradation. Experimental results show that the proposed system assigns the correct phonological representations to unsegmented Japanese sentences more precisely than do other popular systems. Introduction In natural language processing for Japanese text morphological analysis is very important. Currently there are two main methods for automatic part-of-speech tagging namely corpusbased and rule-based methods. The corpus-based method is popular for European languages. Samuelsson and Voutilainen 1997 however show significantly higher achievement of a rulebased tagger than that of statistical taggers for English text. On the other hand most Japanese taggers1 are rule-based. In previous Japanese taggers it was difficult to increase the accuracy of the analysis. Takeuchi and Matsumoto 1995 combined a rule-based and a corpus-based method 1 In this paper a tagger is identical to a morphological analyzer. resulting in a marginal increase in the accuracy of their taggers. However this increase is still insufficient. The source of the trouble is the difficulty in adjusting the grammar and parameters. Our tagger is also rule-based. By using the cooccurrence of words it reduces the difficulty and generates a continuous increase in its accuracy. The proposed system analyzes .

• Báo cáo khoa học
• Japanese Morphological
• Analyzer using Word
• Long Papers
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học