Self-supervised optical flow derotation network for rotation estimation of a spherical camera

摘要

In this paper, we propose a self-supervised optical flow-based approach to learn the rotation of an arbitrarily moving spherical camera. Nowadays, deep learning has enabled efficient learning of camera rotation efficiently. However, most approaches are fully supervised and require large datasets with ground-truth labels of the rotation, and these labels are difficult to acquire. We attempt to solve this problem by using a derotation operation of the spherical optical flow on a unit sphere. This operation decouples the camera rotation from the mixture of translational and rotational components, removing the effect of 3D information for rotation estimation. Therefore, we integrate a derotation layer into a convolutional neural network for regressing the camera rotation. This layer can be adopted for only spherical cameras, which can capture all-round information, and thus enables the network to be learned the camera rotation without using labeled training datasets. We experimentally demonstrate that our approach achieves the comparable performance for the rotation estimation to that of a fully supervised approach and that it outperforms a previously proposed approach. Moreover, transfer learning is conducted in new environments to confirm the benefit of the self-supervised learning.