METHOD FOR UNDERSTANDING AND SYNTHESIZING DIFFERENTIABLE SCENES FROM INPUT IMAGES

摘要

A method for synthesizing new images from an input image comprising: receiving the input image; identifying at least one object in the input image; constructing a scene graph representative of the at least one object; enhancing the scene graph with object properties from a 3D model of the object to produce an enhanced scene representation; rendering a new image from the enhanced scene graph. According to certain embodiments, the method further includes comparing the new rendered image with the input image to compute a loss value representative of a difference between the original image and the new rendered image. The method may further include using the computed loss value to update the object properties in the enhanced scene representation. In some embodiments, the method may include iteratively rendering a new image, computing the loss value for the new image, and updating the object properties in the enhanced scene representation.