ABSTRACT
This chapter introduces the fundamentals of information fusion, reviews the state of the art of fusion strategies, and outlines the issues and challenges that are associated with the design of a multimodal system. It discusses kernel-based fusion methods, and examines their feasibility in an audiovisual-based human emotion recognition problem. The chapter provides a review of multimodal information fusion and related works according to their levels of fusion. Multimedia contain a combination of information content from different media sources in various content forms, such as audio, video, image, and text, each of which can be deemed as a modality in a multimodal multimedia representation. The objective of information fusion is to combine different modalities for accomplishing more accurate pattern analysis and recognition. Many multisensor-based fusion methods have been introduced for target tracking applications. The kernel matrix fusion approach finds a joint subspace for different modalities, and outputs one feature vector.
