tailieunhanh - Báo cáo khoa học: "A Web-based Evaluation Framework for Spatial Instruction-Giving Systems"

We demonstrate a web-based environment for development and testing of different pedestrian route instruction-giving systems. The environment contains a City Model, a TTS interface, a game-world, and a user GUI including a simulated street-view. We describe the environment and components, the metrics that can be used for the evaluation of pedestrian route instruction-giving systems, and the shared challenge which is being organised using this environment. | A Web-based Evaluation Framework for Spatial Instruction-Giving Systems Srinivasan Janarthanam Oliver Lemon and Xingkun Liu Interaction Lab School of Mathematical and Computer Sciences Heriot Watt University Edinburgh sc445 Abstract We demonstrate a web-based environment for development and testing of different pedestrian route instruction-giving systems. The environment contains a City Model a TTS interface a game-world and a user GUI including a simulated street-view. We describe the environment and components the metrics that can be used for the evaluation of pedestrian route instruction-giving systems and the shared challenge which is being organised using this environment. 1 Introduction Generating navigation instructions in the real world for pedestrians is an interesting research problem for researchers in both computational linguistics and geo-informatics Dale et al. 2003 Richter and Duckham 2008 . These systems generate verbal route directions for users to go from A to B and techniques range from giving a priori route directions . all route information in a single turn and incremental in-situ instructions to full interactive dialogue systems see section 4 . One of the major problems in developing such systems is in evaluating them with real users in the real world. Such evaluations are expensive time consuming and painstaking to organise and are carried out not just at the end of the project but also during the development cycle. Consequently there is a need for a common platform to effectively compare the performances of verbal navigation systems developed by different teams using a variety of techniques . a priori vs. in-situ or rule-based vs. machine learning . 49 This demonstration system brings together existing online data resources and software toolkits to create a low-cost framework for evaluation of pedestrian route instruction systems. We have built a web-based environment containing a simulated real world in which .