BabyTalk: understanding and generating simple image descriptions

摘要

Kulkarni et al. propose an automatic image description system based on predicted content using natural language statistics. They claim that the system is able to produce a correct description of an image in the form of natural language sentences, in the same way that humans can. The proposed system contains two stages: computer vision-based detection and recognition following content planning, and the framing of natural language sentences based on predicted content and natural language statistics through surface realization. The paper uses several surface realization steps and evaluates each automatically formed sentence, similar to human interpretation.