tailieunhanh - Báo cáo khoa học: "The Distributional Inclusion Hypotheses and Lexical Entailment"

This paper suggests refinements for the Distributional Similarity Hypothesis. Our proposed hypotheses relate the distributional behavior of pairs of words to lexical entailment – a tighter notion of semantic similarity that is required by many NLP applications. To automatically explore the validity of the defined hypotheses we developed an inclusion testing algorithm for characteristic features of two words, which incorporates corpus and web-based feature sampling to overcome data sparseness. . | The Distributional Inclusion Hypotheses and Lexical Entailment Maayan Geffet School of Computer Science and Engineering Hebrew University Jerusalem Israel 91904 mary@ Abstract This paper suggests refinements for the Distributional Similarity Hypothesis. Our proposed hypotheses relate the distributional behavior of pairs of words to lexical entailment - a tighter notion of semantic similarity that is required by many NLP applications. To automatically explore the validity of the defined hypotheses we developed an inclusion testing algorithm for characteristic features of two words which incorporates corpus and web-based feature sampling to overcome data sparseness. The degree of hypotheses validity was then empirically tested and manually analyzed with respect to the word sense level. In addition the above testing algorithm was exploited to improve lexical entailment acquisition. 1 Introduction Distributional Similarity between words has been an active research area for more than a decade. It is based on the general idea of Harris Distributional Hypothesis suggesting that words that occur within similar contexts are semantically similar Harris 1968 . Concrete similarity measures compare a pair of weighted context feature vectors that characterize two words Church and Hanks 1990 Ruge 1992 Pereira et al. 1993 Grefenstette 1994 Lee 1997 Lin 1998 Pantel and Lin 2002 Weeds and Weir 2003 . As it turns out distributional similarity captures a somewhat loose notion of semantic similarity see Table 1 . It does not ensure that the meaning of one word is preserved when replacing it with the other one in some context. Ido Dagan Department of Computer Science Bar-Ilan University Ramat-Gan Israel 52900 dagan@cs . However many semantic information-oriented applications like Question Answering Information Extraction and Paraphrase Acquisition require a tighter similarity criterion as was also demonstrated by papers at the recent PASCAL Challenge on .