tailieunhanh - Báo cáo khoa học: "Evaluating the Accuracy of an Unlexicalized Statistical Parser on the PARC DepBank"

We evaluate the accuracy of an unlexicalized statistical parser, trained on 4K treebanked sentences from balanced data and tested on the PARC DepBank. We demonstrate that a parser which is competitive in accuracy (without sacrificing processing speed) can be quickly tuned without reliance on large in-domain manuallyconstructed treebanks. This makes it more practical to use statistical parsers in applications that need access to aspects of predicate-argument structure. | Evaluating the Accuracy of an Unlexicalized Statistical Parser on the PARC DepBank Ted Briscoe Computer Laboratory University of Cambridge John Carroll School of Informatics University of Sussex Abstract We evaluate the accuracy of an unlexi-calized statistical parser trained on 4K treebanked sentences from balanced data and tested on the PARC DepBank. We demonstrate that a parser which is competitive in accuracy without sacrificing processing speed can be quickly tuned without reliance on large in-domain manually-constructed treebanks. This makes it more practical to use statistical parsers in applications that need access to aspects of predicate-argument structure. The comparison of systems using DepBank is not straightforward so we extend and validate DepBank and highlight a number of representation and scoring issues for relational evaluation schemes. 1 Introduction Considerable progress has been made in accurate statistical parsing of realistic texts yielding rooted hierarchical and or relational representations of full sentences. However much of this progress has been made with systems based on large lexicalized probabilistic context-free like PCFG-like models trained on the Wall Street Journal WSJ subset of the Penn TreeBank PTB . Evaluation of these systems has been mostly in terms of the PARSEVAL scheme using tree similarity measures of labelled precision and recall and crossing bracket rate applied to section 23 of the WSJ PTB. See . Collins 1999 for detailed exposition of one such very fruitful line of research. We evaluate the comparative accuracy of an un-lexicalized statistical parser trained on a smaller treebank and tested on a subset of section 23 of the WSJ using a relational evaluation scheme. We demonstrate that a parser which is competitive in accuracy without sacrificing processing speed can be quickly developed without reliance on large in-domain manually-constructed treebanks. This makes it more practical to use statistical parsers in .