tailieunhanh - Báo cáo khoa học: "Bilingual Sense Similarity for Statistical Machine Translation"

This paper proposes new algorithms to compute the sense similarity between two units (words, phrases, rules, etc.) from parallel corpora. The sense similarity scores are computed by using the vector space model. We then apply the algorithms to statistical machine translation by computing the sense similarity between the source and target side of translation rule pairs. Similarity scores are used as additional features of the translation model to improve translation performance. | Bilingual Sense Similarity for Statistical Machine Translation Boxing Chen George Foster and Roland Kuhn National Research Council Canada 283 Alexandre-Taché Boulevard Gatineau Quebec Canada J8X 3X7 @ Abstract This paper proposes new algorithms to compute the sense similarity between two units words phrases rules etc. from parallel corpora. The sense similarity scores are computed by using the vector space model. We then apply the algorithms to statistical machine translation by computing the sense similarity between the source and target side of translation rule pairs. Similarity scores are used as additional features of the translation model to improve translation performance. Significant improvements are obtained over a state-of-the-art hierarchical phrase-based machine translation system. 1 Introduction The sense of a term can generally be inferred from its context. The underlying idea is that a term is characterized by the contexts it co-occurs with. This is also well known as the Distributional Hypothesis Harris 1954 terms occurring in similar contexts tend to have similar meanings. There has been a lot of work to compute the sense similarity between terms based on their distribution in a corpus such as Hindle 1990 Lund and Burgess 1996 Landauer and Dumais 1997 Lin 1998 Turney 2001 Pantel and Lin 2002 Pado and Lapata 2007 . In the work just cited a common procedure is followed. Given two terms to be compared one first extracts various features for each term from their contexts in a corpus and forms a vector space model VSM then one computes their similarity by using similarity functions. The features include words within a surface window of a fixed size Lund and Burgess 1996 grammatical dependencies Lin 1998 Pantel and Lin 2002 Pado and Lapata 2007 etc. The similari ty function which has been most widely used is cosine distance Salton and McGill 1983 other similarity functions include Euclidean distance City Block .