Image Based Relighting Using Neural Networks

摘要

We present a neural network regression method for relighting realworldrnscenes from a small number of images. The relighting inrnthis work is formulated as the product of the scene’s light transportrnmatrix and new lighting vectors, with the light transport matrix reconstructedrnfrom the input images. Based on the observation thatrnthere should exist non-linear local coherence in the light transportrnmatrix, our method approximates matrix segments using neural networksrnthat model light transport as a non-linear function of lightrnsource position and pixel coordinates. Central to this approach isrna proposed neural network design which incorporates various elementsrnthat facilitate modeling of light transport from a small imagernset. In contrast to most image based relighting techniques, thisrnregression-based approach allows input images to be captured underrnarbitrary illumination conditions, including light sources movedrnfreely by hand. We validate our method with light transport data ofrnreal scenes containing complex lighting effects, and demonstraternthat fewer input images are required in comparison to related techniques.