SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images

摘要

We study the problem of symmetry detection of 3D shapes from single-viewRGB-D images, where severely missing data renders geometric detectionapproach infeasible. We propose an end-to-end deep neural network whichis able to predict both reflectional and rotational symmetries of 3D objectspresent in the input RGB-D image. Directly training a deep model for symmetryprediction, however, can quickly run into the issue of overfitting. Weadopt a multi-task learning approach. Aside from symmetry axis prediction,our network is also trained to predict symmetry correspondences. Inparticular, given the 3D points present in the RGB-D image, our networkoutputs for each 3D point its symmetric counterpart corresponding to aspecific predicted symmetry. In addition, our network is able to detect fora given shape multiple symmetries of different types. We also contribute abenchmark of 3D symmetry detection based on single-view RGB-D images.Extensive evaluation on the benchmark demonstrates the strong generalizationability of our method, in terms of high accuracy of both symmetryaxis prediction and counterpart estimation. In particular, our method is robust in handling unseen object instances with large variation in shape,multi-symmetry composition, as well as novel object categories.