Image-based relighting using image segmentation and bootstrap strategy

摘要

Image-based relighting technologies enable us to recover the illumination effects of modeled scenes under new light conditions without complicated geometrical information. However, most of them are troubled by specialized devices and tedious sampling work. In this study, we propose an efficient and accurate image-based relighting method for the estimation of the light transport matrix of modeled scene, starting from a small number of images acquired with a fixed viewpoint and with lighting sampled over a uniform 2D grid. Especially, the image space is segmented based on the position and average color value of each pixel usingK-means. The local coherence among the pixels can be considered to associate with pixel position and pixels’ albedo. The pixels of each cluster can be trained by several neural networks and the training scene datasets can be chosen using the bootstrap strategy. These tricks improve the regression performance. We validate our method with light transport data of several scenes containing complex lighting effects. The obtained results show that the proposed method is useful for practical applications and we can get more plausible rendered images with fewer input images in comparison to related techniques.