Đang chuẩn bị liên kết để tải về tài liệu:
Báo cáo khoa học: "A Discriminative Language Model with Pseudo-Negative Samples"
Đang chuẩn bị nút TẢI XUỐNG, xin hãy chờ
Tải xuống
In this paper, we propose a novel discriminative language model, which can be applied quite generally. Compared to the well known N-gram language models, discriminative language models can achieve more accurate discrimination because they can employ overlapping features and nonlocal information. However, discriminative language models have been used only for re-ranking in specific applications because negative examples are not available. | A Discriminative Language Model with Pseudo-Negative Samples Daisuke Okanoharaf Jun ichi Tsujiift f Department of Computer Science University of Tokyo Hongo 7-3-1 Bunkyo-ku Tokyo Japan ị School of Informatics University of Manchester NaCTeM National Center for Text Mining hillbig tsujii @is.s.u-tokyo.ac.jp Abstract In this paper we propose a novel discriminative language model which can be applied quite generally. Compared to the well known N-gram language models discriminative language models can achieve more accurate discrimination because they can employ overlapping features and nonlocal information. However discriminative language models have been used only for re-ranking in specific applications because negative examples are not available. We propose sampling pseudo-negative examples taken from probabilistic language models. However this approach requires prohibitive computational cost if we are dealing with quite a few features and training samples. We tackle the problem by estimating the latent information in sentences using a semiMarkov class model and then extracting features from them. We also use an online margin-based algorithm with efficient kernel computation. Experimental results show that pseudo-negative examples can be treated as real negative examples and our model can classify these sentences correctly. 1 Introduction Language models LMs are fundamental tools for many applications such as speech recognition machine translation and spelling correction. The goal of LMs is to determine whether a sentence is correct or incorrect in terms of grammars and pragmatics. 73 The most widely used LM is a probabilistic language model PLM which assigns a probability to a sentence or a word sequence. In particular Ngrams with maximum likelihood estimation NLMs are often used. Although NLMs are simple they are effective for many applications. However NLMs cannot determine correctness of a sentence independently because the probability depends on the length of .