tailieunhanh - Báo cáo khoa học: "An Ensemble Method for Selection of High Quality Parses"

While the average performance of statistical parsers gradually improves, they still attach to many sentences annotations of rather low quality. The number of such sentences grows when the training and test data are taken from different domains, which is the case for major web applications such as information retrieval and question answering. In this paper we present a Sample Ensemble Parse Assessment (SEPA) algorithm for detecting parse quality. | An Ensemble Method for Selection of High Quality Parses Roi Reichart ICNC Hebrew University of Jerusalem roiri@ Ari Rappoport Institute of Computer Science Hebrew University of Jerusalem arir@ Abstract While the average performance of statistical parsers gradually improves they still attach to many sentences annotations of rather low quality. The number of such sentences grows when the training and test data are taken from different domains which is the case for major web applications such as information retrieval and question answering. In this paper we present a Sample Ensemble Parse Assessment SEPA algorithm for detecting parse quality. We use a function of the agreement among several copies of a parser each of which trained on a different sample from the training data to assess parse quality. We experimented with both generative and reranking parsers Collins Charniak and Johnson respectively . We show superior results over several baselines both when the training and test data are from the same domain and when they are from different domains. For a test setting used by previous work we show an error reduction of 31 as opposed to their 20 . 1 Introduction Many algorithms for major NLP applications such as information extraction IE and question answering QA utilize the output of statistical parsers see Yates et al. 2006 . While the average performance of statistical parsers gradually improves the quality of many of the parses they produce is too low for applications. When the training and test 408 data are taken from different domains the parser adaptation scenario the ratio of such low quality parses becomes even higher. Figure 1 demonstrates these phenomena for two leading models Collins 1999 model 2 a generative model and Charniak and Johnson 2005 a reranking model. The parser adaptation scenario is the rule rather than the exception for QA and IE systems because these usually operate over the highly variable Web making it very .