Understanding and generating scenes using neural networks

摘要

Methods, systems and devices, including computer programs encoded on computer storage media for rendering of images. In one aspect, the method includes receiving a plurality of observations characterizing a particular scene, each observation including an image of the particular scene and data identifying the location of the camera that captured the image. . In another aspect, the method comprises receiving a plurality of observations characterizing a particular video, each observation comprising data identifying a video frame from the specific video and a timestamp of the video frame in the specific video. Including, including steps. In yet another aspect, a method includes receiving a plurality of observations characterizing a particular image, wherein each observation includes a crop of the particular image and data characterizing the crop of the particular image. . The method processes each of the plurality of observations using an observation neural network and determines a numerical representation as an output.