Generation of a Short Narrative Caption for an Image Using the Suggested Hashtag

摘要

Existing methods for image captioning aim to generate captions in natural language by using the image attributes. Though these captions are enough to explain what the image is about in most cases, yet sometimes more than a single sentence is desired to describe the context of an image especially when a good caption for the image draws more public attention or 'likes' on a social media post. Though some work has been done to develop models that can generate hashtags for images, but no research work exists that can use those hashtags to create storylike captions. A hashtag can be defined as a word preceded by the symbol '#' and is used to identify an image on social media sites like Instagram. The goal of this application paper is to explore the possibility of generating hashtags for an input image and leverage it to generate meaningful anecdotes connecting to the essence of the image. The experiment conducted, uses an attention-based encoder-decoder framework to produce hashtags for the raw image while a character-level language model, which is trained using a multi-layer RNN, is used to generate stories using one of the suggested hashtags. The model was then tested on HARRISON dataset of Instagram images and the results were qualitatively analyzed through a user study. After analyzing the outcomes of the experiment, it was concluded that this area of research has huge prospects.