Dynamically estimating lighting parameters for the positions within augmented-reality scenes using a neural network

摘要

The application relates to methods, computer readable media, and systems that use a local-lighting-estimation-neural network to estimate lighting parameters for specific positions within a digital scene for augmented reality. For example, based on a request to render a virtual object in a digital scene, a system uses a local-lighting-estimation-neural network to generate location-specific-lighting parameters for a designated position within the digital scene. In certain implementations, the system also renders a modified digital scene comprising the virtual object at the designated position according to the parameters. In some embodiments, the system generates such location-specific-lighting parameters to spatially vary and adapt lighting conditions for different positions within a digital scene. As requests to render a virtual object come in real (or near real) time, the system can quickly generate different location-specific-lighting parameters that accurately reflect lighting conditions at different positions within a digital scene in response to render requests. A method of training said neural network is also described. The global feature map may be modified by generating a masking feature map from the local position co-ordinates then multiplying the global feature map and the masking feature map to generate a masked-dense-feature map, concatenating the global-feature- map and the masked-dense-feature map and providing this combined map to the neural network to generate the location specific lighting parameters.