Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections

摘要

3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.