Words to Meaningful Sentence: Analytics through Game Interface

摘要

In this work, we designed an experiment to discover the sentence making process followed by a human. To achieve the goal of meaningful sentence construction with a context by the computer, it requires an efficient algorithm. The computer can identify visual elements of a scene through the convolutional neural network. These visual elements are a few important words in a whole sentence. A human being constructs a meaningful sentence with such visual elements by incorporating connecting words which is not yet plausible by the computer. The typical generative deep models generate sentences which many times does not follow the context because the methods of sentence construction by a human (i.e., interpolation of connecting words) are less studied. This research is an attempt to have a better idea of the words interpolation. This will be useful to devise an efficient algorithm for image description generation especially for the useful application of visually impaired or blind people.