LEARNING KEYPOINTS AND MATCHING RGB IMAGES TO CAD MODELS

摘要

A neural network or system can be configured to learn keypoint locations and respective descriptors associated with each keypoint location. The network can include a CAD domain and a picture or RGB-D domain. The CAD domain can include a first branch of the network and a second branch of the network. The CAD domain can be configured to train on pairs of depth images rendered from CAD models of CAD objects, so as to learn viewpoint-invariant features of the CAD objects. The picture domain can include a third branch of the network and a fourth branch of the network. The picture domain can be configured to train on pairs of images of objects, for instance a depth image and its corresponding RGB image, so as to learn modularity-invariant features of the objects. At test time, the network can identify an RGB image, for instance a pose or category defined by the RGB image, using a database created from depth images rendered from CAD models.