Learning visual similarity for product design with convolutional neural networks

摘要

Popular sites like Houzz, Pinterest, and LikeThatDecor, have communitiesrnof users helping each other answer questions about productsrnin images. In this paper we learn an embedding for visual search inrninterior design. Our embedding contains two different domains ofrnproduct images: products cropped from internet scenes, and productsrnin their iconic form. With such a multi-domain embedding, werndemonstrate several applications of visual search including identifyingrnproducts in scenes and finding stylistically similar products. Tornobtain the embedding, we train a convolutional neural network onrnpairs of images. We explore several training architectures includingrnre-purposing object classifiers, using siamese networks, and usingrnmultitask learning. We evaluate our search quantitatively and qualitativelyrnand demonstrate high quality results for search across multiplernvisual domains, enabling new applications in interior design.