A Comprehensive Survey on Multimodal Data Representation and Information Fusion Algorithms

摘要

A contemporary survey on recent advancements in the field of multimodal signal processing, with a focus on multimodal data representation and information fusion is presented in this paper. Multimodal data representation is of critical importance in many signal processing applications, and information fusion algorithms aim at narrowing the heterogeneity gap among the different modalities. First, we start with a brief overview on techniques with some of the commonly used unimodal signals such as text, speech and image, which serves as fundamental requirement in multimodal representation. Next, we discuss multimodal data representation with audio-video, iris, fingerprint, face, LiDAR scanning and images. Later, we provide details on information fusion, broadly classified into model-agnostic and model-based approaches and mention some applications. Further, we discuss some of the challenges associated with multimodal signal processing, in terms of uncertainties, mismatches and inaccuracies in data representation and fusion.