tailieunhanh - Báo cáo khoa học: "A text-based search interface for Multimedia Dialectics"

The growing popularity of multimedia documents requires language technologies to approach automatic language analysis and generation from yet another perspective: that of its use in multimodal communication. In this paper, we present a support tool for COSMOROE, a theoretical framework for modelling multimedia dialectics. The tool is a text-based search interface that facilitates the exploration of a corpus of audiovisual files, annotated with the COSMOROE relations. | A text-based search interface for Multimedia Dialectics Katerina Pastra Inst. for Language Speech Processing Athens Greece kpastra@ Eirini Balta Inst. for Language Speech Processing Athens Greece ebalta@ Abstract The growing popularity of multimedia documents requires language technologies to approach automatic language analysis and generation from yet another perspective that of its use in multimodal communication. In this paper we present a support tool for COSMOROE a theoretical framework for modelling multimedia dialectics. The tool is a text-based search interface that facilitates the exploration of a corpus of audiovisual files annotated with the COSMOROE relations. 1 Introduction Online multimedia content becomes more and more accessible through digital TV social networking sites and searchable digital libraries of photographs and videos. People of different ages and cultures attempt to make sense out of this data and re-package it for their own needs these being informative educational and entertainment ones. Understanding and generation of multimedia discourse requires knowledge and skills related to the nature of the interacting modalities and their semantic interplay for formulating the multimedia message. Within such context intelligent multimedia systems are expected to parse generate such messages or at least assist humans in these tasks. From another perspective everyday human communication is predominantly multimodal as such similarly intuitive human-computer robot interaction demands that intelligent systems master among others the semantic interplay between different media and modalities . they are able to use understand natural language and its reference to objects and activities in the shared situated communication space. It was more than a decade ago when the lack of a theory of how different media interact with one another was indicated Whittaker and Walker 1991 . Recently such theoretical framework has been developed and used