Weakly Supervised Learning of Image Emotion Analysis Based on Cross-spatial Pooling

摘要

Convolutional neural networks (CNNs) simulate the structure and function of the nervous system based on biological characteristics. CNNs have been used to understand the emotions that images convey. Most existing studies of emotion analysis have focused only on image emotion classification, and few studies have paid attention to relevant regions evoking emotions. In this paper, we solve the issues of image emotion classification and emotional region localization based on weakly supervised deep learning in a unified framework. We train a fully convolutional network, followed by our proposed cross-spatial pooling strategy, to generate an emotional activation map (EAM), which represents the relevant region that could evoke emotion in an image and is only labelled with an image-level annotation. Extensive experiments demonstrate that our proposed method has the best performance in the accuracy of classification and emotional region localization.