tailieunhanh - Báo cáo khoa học: "A Test Environment for Natural Language Understanding Systems"

The Natural Language Understanding Engine Test Environment (ETE) is a GUI software tool that aids in the development and maintenance of large, modular, natural language understanding (NLU) systems. Natural language understanding systems are composed of modules (such as partof-speech taggers, parsers and semantic analyzers) which are difficult to test individually because of the complexity of their output data structures. Not only are the output data structures of the internal modules complex, but also many thousands of test items (messages or sentences) are required to provide a reasonable sample of the linguistic structures of a single human language, even if. | A Test Environment for Natural Language Understanding Systems Li Li Deborah A. Dahl Lewis M. Norton Marcia c. Linebarger Dongdong Chen Unisys Corporation 2476 Swedesford Road Malvern PA 19355 . @ Abstract The Natural Language Understanding Engine Test Environment ETE is a GUI software tool that aids in the development and maintenance of large modular natural language understanding NLU systems. Natural language understanding systems are composed of modules such as part-of-speech taggers parsers and semantic analyzers which are difficult to test individually because of the complexity of their output data structures. Not only are the output data structures of the internal modules complex but also many thousands of test items messages or sentences are required to provide a reasonable sample of the linguistic structures of a single human language even if the language is restricted to a particular domain. The ETE assists in the management and analysis of the thousands of complex data structures created during natural language processing of a large corpus using relational database technology in a network environment. Introduction Because of the complexity of the internal data structures and the number of test cases involved in testing a natural language understanding system evaluation of testing results by manual comparison of the internal data structures is very difficult. The difficulty of examining NLU systems in turn greatly increases the difficulty of developing and extending the coverage of these systems both because as the system increases in coverage and complexity extensions become progressively harder to assess and because loss of coverage of previously working test data becomes harder to detect. The ETE addresses these problems by 1. managing batch input of large numbers of test sentences or messages whether spoken or written. 2. storing the NLU system output for a batch run into a .