tailieunhanh - Báo cáo khoa học: "PLANNING MULTIMODAL DISCOURSE "

In this talk, we will, show how techniques for planning text and discourse can be generalized to plan the structure and content of multimodal communications, that integrate natural language, pointing, graphics, and animations. The central claim of this talk is that the generation of multimodal discourse can be considered as an incremental planning process that aims to achieve a given communicative goal. One of the surprises from our research is that it is actually possible to extend and adapt many of the fundamental concepts developed to date in computatational linguistics in such a way that they become useful for. | PLANNING MULTIMODAL DISCOURSE Wolfgang Wahlster German Research Center for Artificial Intelligence DFKI Stuhlsatzenhausweg 3 D-6600 Saarbriicken 11 Germany Internet wahlster@ Abstract In this talk we will show how techniques for planning text and discourse can be generalized to plan the structure and content of multimodal communications that integrate natural language pointing graphics and animations. The central claim of this talk is that the generation of multimodal discourse can be considered as an incremental planning process that aims to achieve a given communicative goal. One of the surprises from our research is that it is actually possible to extend and adapt many of the fundamental concepts developed to date in computatational linguistics in such a way that they become useful for multimodal discourse as well. This means that an interesting methodological transfer from the area of natural language processing to a much broader computational model of multimodal communication is possible. In particular semantic and pragmatic concepts like speech acts coherence focus communicative act discourse model reference implicature anaphora rhetorical relations and scope ambiguity take an extended meaning in the context of multimodal discourse. It is an important goal of this research not simply to merge the verbalization and visualization results of mode-specific generators but to carefully coordinate them in such a way that they generate a multiplicative improvement in communication capabilities. Allowing all of the modalities to refer to and depend upon each other is a key to the richness of multimodal communication. A basic principle underlying our model is that the various constituents of a multimodal communication should be generated from a common representation of what is to be conveyed. This raises the question of how to decompose a given communicative goal into subgoals to be realized by the mode-specific generators so that they complement each .