The efficient method of mixed reality light restoration using HDR image of 3D scene

摘要

One of the main problems of MR devices is a physically correct representation of luminance distribution for virtual objects and its shadows in the real world. In other words, reconstruction of the luminance distribution over the scene is one of the key parameters that allow solving the problem of "seamless" interaction between the virtual and real worlds and serves as the primary means for improving the quality of perception when we use the augmented reality system. By using neuro networks, it becomes possible to estimate the number of light sources, their positions and radiation gonio - diagrams. Authors use the fast and high-quality photorealistic image synthesis to synthesize the virtual image of the real world under the restored illumination conditions. Having both real world image and synthesized real world image at restored illumination conditions it becomes possible to estimate the error of the roughly estimated light source positions to correct it. It is possible by application of the special shading analysis method. The first step requires two kind of information: reconstructed geometry of surrounding space and HDR image, with luminance data. One of the main issues is that there are two kinds of illumination: primary illumination, from light sources, and secondary illumination or background illumination, from non-luminous objects. The task is to remove background illumination and define primary illumination. After restoration illumination condition, a virtual object could be properly added to visible image of real environment.