tailieunhanh - Báo cáo khoa học: "Bi-Directional Parsing for Generic Multimodal Interaction"

We introduce a new multi-threaded parsing algorithm on unification grammars designed specifically for multimodal interaction and noisy environments. By lifting some traditional constraints, namely those related to the ordering of constituents, we overcome several difficulties of other systems in this domain. We also present several criteria used in this model to constrain the search process using dynamically loadable scoring functions. Some early analyses of our implementation are discussed. . | Clavius Bi-Directional Parsing for Generic Multimodal Interaction Frank Rudzicz Centre for Intelligent Machines McGill University Montreal Canada frudzi@ Abstract We introduce a new multi-threaded parsing algorithm on unification grammars designed specifically for multimodal interaction and noisy environments. By lifting some traditional constraints namely those related to the ordering of constituents we overcome several difficulties of other systems in this domain. We also present several criteria used in this model to constrain the search process using dynamically loadable scoring functions. Some early analyses of our implementation are discussed. 1 Introduction Since the seminal work of Bolt Bolt 1980 the methods applied to multimodal interaction MMI have diverged towards unreconcilable approaches retrofitted to models not specifically amenable to the problem. For example the representational differences between neural networks decision trees and finite-state machines Johnston and Bangalore 2000 have limited the adoption of the results using these models and the typical reliance on the use of whole unimodal sentences defeats one of the main advantages of MMI - the ability to constrain the search using cross-modal information as early as possible. Clavius is the result of an effort to combine sensing technologies for several modality types speech and video-tracked gestures chief among them within the immersive virtual environment Boussemart 2004 shown in Figure 1. Its purpose is to comprehend multimodal phrases such as put this here . for pointing gestures in either command-based or dialogue interaction. Clavius provides a flexible and trainable new bi-directional parsing algorithm on multidimensional input spaces and produces modalityindependent semantic interpretation with a low computational cost. Figure 1 The target immersive environment. Graphical Models and Unification Unification grammars on typed directed acyclic graphs have been explored