RepeatPadding: Balancing words and sentence length for language comprehension in visual question answering

摘要

Visual question answering (VQA) is a complicated Turing-AI task which needs not only to understand the multi-modality inputs but also reason to provide correct answer. Nowadays, there are complicated and sophisticated modules for reasoning in popular works. However, the language representation which is frequently treated as the guider of VQA hasn't been fully explored in current researches, leading to insufficient reasoning and unsatisfactory answer. In this work, two types of method including VieAns and Repeat-Padding which focus on language processing are proposed to balance the sentence by cropping and padding the question, where the language information is transformed to different expressions and further pushes the language model to grab more representative features for further boosting the accuracy of predicted answers. Experiments on the benchmark COCO-QA and VQA2.0 datasets are conducted to demonstrate the effectiveness of the proposed method. Particularly, the proposed RepeatPadding method is more suitable for different language models. (C) 2020 Elsevier Inc. All rights reserved.