tailieunhanh - Báo cáo khoa học: "Ask not what Textual Entailment can do for You.."

We challenge the NLP community to participate in a large-scale, distributed effort to design and build resources for developing and evaluating solutions to new and existing NLP tasks in the context of Recognizing Textual Entailment. We argue that the single global label with which RTE examples are annotated is insufficient to effectively evaluate RTE system performance; to promote research on smaller, related NLP tasks, we believe more detailed annotation and evaluation are needed, and that this effort will benefit not just RTE researchers, but the NLP community as a whole. . | Ask not what Textual Entailment can do for You. Mark Sammons Vydiswaran Dan Roth University of Illinois at Urbana-Champaign mssammon vgvinodv danr @ Abstract We challenge the NLP community to participate in a large-scale distributed effort to design and build resources for developing and evaluating solutions to new and existing NLP tasks in the context of Recognizing Textual Entailment. We argue that the single global label with which RTE examples are annotated is insufficient to effectively evaluate RTE system performance to promote research on smaller related NLP tasks we believe more detailed annotation and evaluation are needed and that this effort will benefit not just RTE researchers but the NLP community as a whole. We use insights from successful RTE systems to propose a model for identifying and annotating textual inference phenomena in textual entailment examples and we present the results of a pilot annotation study that show this model is feasible and the results immediately useful. 1 Introduction Much of the work in the field of Natural Language Processing is founded on an assumption of semantic compositionality that there are identifiable separable components of an unspecified inference process that will develop as research in NLP progresses. Tasks such as Named Entity and coreference resolution syntactic and shallow semantic parsing and information and relation extraction have been identified as worthwhile tasks and pursued by numerous researchers. While many have nearly immediate application to real world tasks like search many are also motivated by their potential contribution to more ambitious Natural Language tasks. It is clear that the compo-nents tasks identified so far do not suffice in them selves to solve tasks requiring more complex reasoning and synthesis of information many other tasks must be solved to achieve human-like performance on tasks such as Question Answering. But there is no clear process for identifying