Đang chuẩn bị liên kết để tải về tài liệu:
Báo cáo khoa học: "Instance-based Evaluation of Entailment Rule Acquisition"

Obtaining large volumes of inference knowledge, such as entailment rules, has become a major factor in achieving robust semantic processing. While there has been substantial research on learning algorithms for such knowledge, their evaluation methodology has been problematic, hindering further research. We propose a novel evaluation methodology for entailment rules which explicitly addresses their semantic properties and yields satisfactory human agreement levels. The methodology is used to compare two state of the art learning algorithms, exposing critical issues for future progress. . | Instance-based Evaluation of Entailment Rule Acquisition Idan Szpektor Eyal Shnarch Ido Dagan Dept. of Computer Science Bar Ilan University Ramat Gan Israel szpekti shey dagan @cs.biu.ac.il Abstract Obtaining large volumes of inference knowledge such as entailment rules has become a major factor in achieving robust semantic processing. While there has been substantial research on learning algorithms for such knowledge their evaluation methodology has been problematic hindering further research. We propose a novel evaluation methodology for entailment rules which explicitly addresses their semantic properties and yields satisfactory human agreement levels. The methodology is used to compare two state of the art learning algorithms exposing critical issues for future progress. 1 Introduction In many NLP applications such as Question Answering QA and Information Extraction IE it is crucial to recognize that a particular target meaning can be inferred from different text variants. For example a QA system needs to identify that Aspirin lowers the risk of heart attacks can be inferred from Aspirin prevents heart attacks in order to answer the question What lowers the risk of heart attacks . This type of reasoning has been recognized as a core semantic inference task by the generic textual entailment framework Dagan et al. 2006 . A major obstacle for further progress in semantic inference is the lack of broad-scale knowledgebases for semantic variability patterns Bar-Haim et al. 2006 . One prominent type of inference knowledge representation is inference rules such as para 456 phrases and entailment rules. We define an entailment rule to be a directional relation between two templates text patterns with variables e.g. X prevent Y X lower the risk of Y . The lel t-hand-side template is assumed to entail the right-handside template in certain contexts under the same variable instantiation. Paraphrases can be viewed as bidirectional entailment rules. Such rules capture basic