Action planning for interactive visual scene understanding based on knowledge confidence defined on latent spaces

摘要

This report proposes a method for action planning in a system of interactive visual scene understanding through the use of system knowledge and its confidence. The knowledge confidence is denned as the combination of the following two properties on the latent space of a topic model connecting image features and text labels: 1) Sim­ilarity between an input sample and training samples on the latent space, and 2) the overall associability between each text label as determined by the content of the training samples. We evaluate the proposed method in the context of annotation accuracy and effort for providing answers from users. The experimental results with PASCAL VOC2008 dataset indicate that our proposed method achieved comparable or better annotation accuracy with less effort compared with strategies of 1) always asking the name of objects and 2) generating random questions.