IMAGE CAPTIONING USING WEAK SUPERVISION AND SEMANTIC NATURAL LANGUAGE VECTOR SPACE

摘要

#$%^&*AU2016256753A120170727.pdf#####In a digital media environment to facilitate management of image collections using one or more computing devices, a method to automatically generate image captions using weak supervision data comprising obtaining a target image for caption analysis; applying feature extraction to the target image to generate global concepts corresponding to the image; comparing the target image to images from a source of weakly annotated images to identify visually similar images; building a collection of keywords for the target image indicative of image details by extracting the keywords from the visually similar images; and supplying the collection of keywords indicative of image details as the weak supervision data for caption generation along with the global concepts.Inventors: Wang et al. Title: Image Captioning with Weak Supervision 600 602 Obtain a target image for caption analysis 604 Apply feature extraction to the target image to generate global concepts corresponding to the target image 606 Compare the target image to images from a source of weakly annotated images to identify visually similar images 608 Build a collection of keywords for the target image by extracting the keywords from the visually similar images 610 Supply the collection of keywords for caption generation along with the global concepts. 612 Generate a caption for the target image using the collection of keywords to modulate word weights applied for sentence construction 7a,6