Do Distributed Semantic Models Dream of Electric Sheep? Visualizing Word Representations through Image Synthesis

摘要

We introduce the task of visualizing distributed semantic representations by generating images from word vectors. Given the corpus-based vector encoding the word broccoli, we convert it to a visual representation by means of a cross-modal mapping function, and then use the mapped representation to generate an image of broccoli as "dreamed" by the distributed model. We propose a baseline dream synthesis method based on averaging pictures whose visual representations are topologi-cally close to the mapped vector. Two experiments show that we generate dreams that generally belong to the the right semantic category, and are sometimes accurate enough for subjects to distinguish the intended concept from a related one.