Image Emotion Caption Based on Visual Attention Mechanisms

摘要

Nowadays, most image automatic descriptions are based on objective descriptions and not contain subjective emotions. However, when people observe images, they will have certain subjective feelings. This paper proposes an image emotion description system based on attention mechanism, which automatically generates captions with sentiments. In the encoding stage, we use CNN to extract image features. In the decoding description phase, we use two same direction parallel LSTMs‐ — one represents factual descriptions; another specializes in descriptions with emotion. When attention mechanism detects different regions of the image, whether emotional words need to be output is selected by a switching mechanism. This system makes the image description more vivid and ensures the accuracy of the description.