tailieunhanh - Báo cáo khoa học: "Multimodal Generation in the COMIC Dialogue System"

We describe how context-sensitive, usertailored output is specified and produced in the COMIC multimodal dialogue system. At the conference, we will demonstrate the user-adapted features of the dialogue manager and text planner. three-dimensional walkthrough of the finished bathroom. We will focus on how context-sensitive, usertailored output is generated in the third, guidedbrowsing phase of the interaction. Figure 2 shows a typical user request and response from COMIC in this phase. The pitch accents and multimodal actions are indicated; there is also facial emphasis corresponding to the accented words. . | Multimodal Generation in the COMIC Dialogue System Mary Ellen Foster and Michael White Institute for Communicating and Collaborative Systems School of Informatics University of Edinburgh @ Andrea Setzer and Roberta Catizone Natural Language Processing Group Department of Computer Science University of Sheffield @ Abstract We describe how context-sensitive user-tailored output is specified and produced in the COMIC multimodal dialogue system. At the conference we will demonstrate the user-adapted features of the dialogue manager and text planner. 1 Introduction COMIC1 is an EU IST 5th Framework project combining fundamental research on human-human interaction with advanced technology development for multimodal conversational systems. The project demonstrator system adds a dialogue interface to a CAD-like application used in bathroom sales situations to help clients redesign their rooms. The input to the system includes speech handwriting and pen gestures the output combines synthesised speech a talking head and control of the underlying application. Figure 1 shows screen shots of the COMIC interface. There are four main phases in the demonstrator. First the user specifies the shape of their own bathroom using a combination of speech input pen-gesture recognition and handwriting recognition. Next the user chooses a layout for the sanitary ware in the room. After that the system guides the user in browsing through a range of tiling options for the bathroom. Finally the user is given a 1COnversational Multimodal Interaction with Computers http comic . three-dimensional walkthrough of the finished bathroom. We will focus on how context-sensitive user-tailored output is generated in the third guided-browsing phase of the interaction. Figure 2 shows a typical user request and response from COMIC in this phase. The pitch accents and multimodal actions are indicated there is also facial .