tailieunhanh - Báo cáo khoa học: "User studies and the design of Natural Language Systems"

This paper presents a critical discussion of the various approaches that have been used in the evaluation of Natural Language systems. We conclude that previous approaches have neglected to evaluate systems in the context of their use, . solving a task requiring data retrieval. This raises questions about the validity of such approaches. In the second half of the paper, we report a laboratory study using the Wizard of Oz technique to identify NL requirements for carrying out this task. We evaluate the demands that task dialogues collected using this technique, place upon a prototype Natural Language system. We. | User studies and the design of Natural Language Systems Steve Whittaker and Phil Stenton Hewlett-Packard Laboratories Filton Road Bristol BS12 6QZ UK. email sjw@ Abstract This paper presents a critical discussion of the various approaches that have been used in the evaluation of Natural Language systems. We conclude that previous approaches have neglected to evaluate systems in the context of their use . solving a task requiring data retrieval. This raises questions about the validity of such approaches. In the second half of the paper we report a laboratory study using the Wizard of Oz technique to identify NL requirements for carrying out this task. We evaluate the demands that task dialogues collected using this technique place upon a prototype Natural Language system. We identify three important requirements which arose from the task that we gave our subjects operators specific to the task of database access complex contextual reference and reference to the structure of the information source. We discuss how these might be satisfied by future Natural Language systems. 1 Introduction Approaches to the evaluation of NL systems It is clear that a number of different criteria might be employed in the evaluation of Natural Language NL systems. It is also clear that there is no consensus on how evaluation should be carried out RQR 88 GM84 . Among the different criteria that have been suggested are a Coverage b Learnability c General software requirements d Comparison with other interface media. Coverage is concerned with the set of inputs which the system should be capable of handling and one issue we will discuss is how this set should be identified. Learnability is premised on the fact that complete coverage is not forseeable in the near future. As a consequence any NL system will have limitations and one problem for users will be to learn to communicate within such limitations. Learnability is measured by the ease with which new users are .

• Báo cáo khoa học
• User studies
• the design of Natural Language Systems
• Long Papers
• scientific reports
• model language
• process natural language
• Lecture Human Computer interaction
• Bài giảng Tương tác Người Máy tính
• Human Computer interaction
• Qualitative research techniques
• Ethnographic field studies
• User centered design approaches
• BMC Bioinformatics
• DNA methylation
• Microarray data analysis
• Statistical power
• Sample size calculation
• Bioconductor package
• sales restrictions
• between heterogeneity
• Allergen Bioinformatics
• Recent Trends
• Developments
• Family Based Studies
• Dialog Control
• Natural Language System
• Dialog Control in a Natural Language System
• Expanding the Horizons
• Natural Language Interfaces
• Phil Hayes
• PROSPECTS OR PRACTICAL
• NATURAL LANGUAGE SYSTEMS
• Workshops
• NATURAL LANGUAGE
• INTERACTION WITH MACHINES
• COMPUTER INTEBFACE DESIGN MURRAY TUROFF DEPARTMENT OF COMPUTER
• IiVFORMATION SCIENCE IIEW JERSEY INSTITUTE OF TECHNOLOGY
• EVALUATION OF NATURAL LANGUAGE
• INTERFACES TO DATABASE SYSTEMS
• A PANEL DISCUSSION
• database query system
• J
• Norwood Crout
• INTERFACES TO DATA BASE SYSTEMS
• Bozena Henisz
• INTERPRETING NATURAL LANGUAGE
• DATABASE UPDATES
• S
• Jermld Kaplan Jim David
• TRANSPORTABLE NATURAL LANGUAGE
• INTERFACES TO DATABASES
• Gary G
• Hendrlx and William H
• Lewis
• DATABASES THEORETICAL
• TECHNICAL ISSUES
• TRANSPORTABLE NATURAL LANGUAGE INTERFACES
• PROBLEMS AND TECHNIQUES
• Barbara J
• Grosz
• THEORETICAL TECHNICAL ISSUES
• NATURAL LANGUAGE ACCESS TO DATABASES
• R
• Petrick
• Domain Independent
• Database Access Systems
• ISSUES IN NATURAL LANGUAGE
• ACCESS TO DATABASES
• A LOGIC PROGRAMMING PERSPECTIVE
• Sharon C
• Salveter David Maier
• PLANNING NATURAL LANGUAGE
• EXPRESSIONS REFERRING
• Douglas E
• Appelt
• THE TEXT SYSTEM
• NATURAL LANGUAGE GENERATION
• Kathleen R
• M
• MENTING A DATABASE
• KNOWLEDGE REPRESENTATION
• THE KEY TO THE SELECTION PROBLEM
• E
• Jeffrey Conklin David D
• McDonald
• A KNOWLEDGE ENGINEERING
• NATURAL LANGUAGE UNDERSTANDING
• Stuart C
• Shapiro
• NATURAL LANGUAGE TEXTS
• NECESSARILY GRAMMATICAL
• EVEN COMPLETE
• UNGRAMHATICALITY
• EXTRA GRAMMATICALITY
• NATURAL LANGUAGE UNDERSTANDING SYSTEMS
• Taxonomy
• Descriptions
• Individuals in Natural
• Language Understanding
• The Use of Ooject Special Knowledge
• Natural Language Processing
• Mark H
• Bursteln