An Abstract Model of Multimodal Fusion using Fuzzy Sets to Derive Interactive Emotions

摘要

Recently, social robotics applications have attracted attention, as they address profound life demands. These intelligent robotic applications should be able to perceive, recognize, and respond to human emotional states. Humans express their emotions both verbally through speech and silence and nonverbally through facial expressions and gestures. Although there are several approaches that have been proposed to recognize a limited amount of human emotion based on one modality, limited ad hoc works have been done to integrate and fuse several modalities. In this paper, we propose an abstract model of multimodal fusion of facial expressions, speech and gestures. An abstract semantic model is presented identifying emotions that involves set-theoretic operations and functional mapping. A semantic algebra has been described.