Learning meta-knowledge for few-shot image emotion recognition

摘要

Previous studies have demonstrated that images are of great importance in attracting people's attention and motivating them to take action. Various attributes (e.g., colors, aesthetics, and embedded objects) related to images are considered driving factors. Among which emotions in images, in particular, play a critical role in stimulating individuals, based on the Stimulus-Organism-Response theory. Consequently, many researchers put great efforts to understand image emotions, ranging from developing theoretical models to a broad spectrum of applications. Due to the complex and unstructured characteristics of images, identifying image emotions is challenging. Although some significant progress in image emotion classification has been achieved, inherent constraints still remain unaddressed. For example, acquiring a sufficiently large amount of labeled data to train a good model is costly and inevitably requires lots of human efforts. Besides, building a generalized model applicable to different datasets still needs a deep exploration. Image emotions are very subjective, which also makes such a classification task difficult. This paper proposes a general meta-learning framework for the few-shot image emotion classification, called Meta-IEC. Meta-IEC provides the capability of: (i) adapting to a similar dataset but new classes that have not been encountered before, and (ii) generalizing to a completely different dataset where emotion classes are unseen in the training dataset and only very few labeled images are available. Meta-IEC is also able to capture the uncertainty and ambiguity during the meta-testing, where we implement a hierarchical Bayesian graphical model to understand latent relationships among various parameters between meta-training and meta-testing. Extensive experiments on three commonly used datasets empirically demonstrate the superiority of our method over several state-of-the-art baselines. For example, our meta-learning based model can achieve performance improvement up to 5+%. We also provide some managerial implications on parameter sensitivity and label selection of meta-training and meta-testing.