Unsupervised Texture Transfer from Images to Model Collections

摘要

Large 3D model repositories of common objects are now ubiquitousrnand are increasingly being used in computer graphics and computerrnvision for both analysis and synthesis tasks. However, images ofrnobjects in the real world have a richness of appearance that thesernrepositories do not capture, largely because most existing 3D modelsrnare untextured. In this work we develop an automated pipelinerncapable of transporting texture information from images of realrnobjects to 3D models of similar objects. This is a challengingrnproblem, as an object’s texture as seen in a photograph is distortedrnby many factors, including pose, geometry, and illumination. Theserngeometric and photometric distortions must be undone in orderrnto transfer the pure underlying texture to a new object — the 3Drnmodel. Instead of using problematic dense correspondences, wernfactorize the problem into the reconstruction of a set of base texturesrn(materials) and an illumination model for the object in the image.rnBy exploiting the geometry of the similar 3D model, we reconstructrncertain reliable texture regions and correct for the illumination, fromrnwhich a full texture map can be recovered and applied to the model.rnOur method allows for large-scale unsupervised production of richlyrntextured 3D models directly from image data, providing high qualityrnvirtual objects for 3D scene design or photo editing applications, asrnwell as a wealth of data for training machine learning algorithms forrnvarious inference tasks in graphics and vision.