Estimating 3D Objects from 2D Images using 3D Transformation Network

摘要

Imagining the 3D representation from the projected 2D images based on the knowledge learned on 3D objects is the natural capability of humans even though this involves one-to-many relationships. In this paper, we propose a 2D to 3D cyclic transformation network that can generate a typical 3D representation of the given 2D image and vice versa by training. This network is composed of two cross-domain generators, and two same-domain generators configured in a general generative adversarial framework. The features formed in the latent space of the same-domain generators are fed to the discriminator. The cross-domain generators transform the input to the required cross-domain outputs while the same-domain generators, on the other hand, render stability to the training of the network. Extensive experiments are conducted on the ModelNet40 dataset that demonstrates the effectiveness of the proposed approach.