Radiometric Scene Decomposition: Estimating Complex Reflectance and Natural Illumination from Images.

摘要

The phrase, "a picture is worth a thousand words", is often used to emphasize the wealth of information encoded into an image. While much of this information (e.g., the identities of people in an image, the type and number of objects in an image, etc.) is readily inferred by humans, fully understanding an image is still extremely difficult for computers. One important set of information encoded into images are radiometric scene properties---the properties of a scene related to light. Each pixel in an image indicates the amount of light received by the camera after being reflected, transmitted, or emitted by objects in a scene. It follows that we can learn about the objects of the scene and the scene itself through the image by thinking about the interaction between light and geometry in a scene.;The appearance of objects in an image is primarily due to three factors: the geometry of the scene, the reflectance of the surfaces, and the incident illumination of the scene. Recovering these hidden properties of scenes can give us a deep understanding of a scene. For example, the reflectance of a surface can give a hint at the material properties of that surface. In this thesis, we address the question of how to recover complex, spatially-varying reflectance functions and natural illumination in real scenes from one or more images with known or approximately-known geometry.;Recovering latent radiometric properties from images is difficult because of the severe underdetermined nature of the problem (i.e., there are many potential combinations of reflectance, light, and geometry that would produce identical input images) combined with the overwhelming dimensionality of the problem. In the real world, reflectance functions are complex, requiring many parameters to accurately model. An important aspect of solving this problem is to create a compact mathematical model to express a wide range of surface reflectance. We must also carefully model scene illumination, which typically exhibits complex behavior as well. Prior work has often simply assumed the light incident to a scene is made up of one or more infinitely-distant point lights. This assumption, however, rarely holds up in practice as not only are scenes illuminated by every possible direction, they are also illuminated by other objects interreflecting one another. To accurately infer reflectance and illumination of real-world scenes, we must account for the real-world behavior of reflectance and illumination.;In this work, we develop a mathematical framework for the inference of complex, spatially-varying reflectance and natural illumination in real-world scenes. We use a Bayesian approach, where the radiometric properties (i.e., reflectance and illumination) to be inferred are modeled as random variables. We can then apply statistical priors to model how reflectance and illumination often exist in the real world to help combat the ambiguities created through the image formation process. We use our framework to infer the reflectance and illumination in a variety of scenes, ultimately using it in unrestricted real-world scenes. We show that the framework is capable of recovering complex reflectance and natural illumination in the real world.