The Euclidean embedding learning based on convolutional neural network for stereo matching

摘要

Stereo matching is one of the most important and fundamental topics in computer vision. The calculation of matching cost plays a very important role for stereo matching algorithms. The stereo matching algorithm proposed by Zbontar and LeCun focusing on the training of the matching cost has showed the good performance of the convolutional neural network. Unfortunately, computing a convolutional neural network for matching cost is computationally very expensive. This paper proposes a method based on learning a Euclidean embedding using a convolutional neural network with a triplet-based loss function, where the matching cost is directly computed by the squared L2 distances between two vectors in the embedding space. The cost is refined by Semiglobal Matching with an adaptive smoothness constraint based on multi-scale segmentations. The proposed method has a comparable performance with the state-of-the-art algorithms, and it overcomes a problem of heavy computation. The proposed method takes only about 5 s for predicting a single image pair, where the computing of convolutional neural networks needs less than 2 s with CPU, that is much faster than the algorithm by Zbontar and LeCun where the computing of convolutional neural network takes 67 s with GPU. (C) 2017 Elsevier B.V. All rights reserved.