tailieunhanh - Báo cáo hóa học: "Joint modality fusion and temporal context exploitation for semantic video analysis"

Tuyển tập các báo cáo nghiên cứu về hóa học được đăng trên tạp chí sinh học đề tài :Joint modality fusion and temporal context exploitation for semantic video analysis | Papadopoulos et al. EURASIP Journal on Advances in Signal Processing 2011 2011 89 http content 2011 1 89 o EURASIP Journal on Advances in Signal Processing a SpringerOpen Journal RESEARCH Open Access Joint modality fusion and temporal context exploitation for semantic video analysis 1 1 Georgios Th Papadopoulos Vasileios Mezaris loannis Kompatsiaris and Michael G Strintzis Abstract In this paper a multi-modal context-aware approach to semantic video analysis is presented. Overall the examined video sequence is initially segmented into shots and for every resulting shot appropriate color motion and audio features are extracted. Then Hidden Markov Models HMMs are employed for performing an initial association of each shot with the semantic classes that are of interest separately for each modality. Subsequently a graphical modeling-based approach is proposed for jointly performing modality fusion and temporal context exploitation. Novelties of this work include the combined use of contextual information and multi-modal fusion and the development of a new representation for providing motion distribution information to HMMs. Specifically an integrated Bayesian Network is introduced for simultaneously performing information fusion of the individual modality analysis results and exploitation of temporal context contrary to the usual practice of performing each task separately. Contextual information is in the form of temporal relations among the supported classes. Additionally a new computationally efficient method for providing motion energy distribution-related information to HMMs which supports the incorporation of motion characteristics from previous frames to the currently examined one is presented. The final outcome of this overall video analysis framework is the association of a semantic class with every shot. Experimental results as well as comparative evaluation from the application of the proposed approach to four datasets .