Depth image upsampling based on guided filter with low gradient minimization

摘要

In this paper, we present a novel upsampling framework to enhance the spatial resolution of the depth image. In our framework, the upscaling of a low-resolution depth image is guided by a corresponding intensity images; we formulate it as a cost aggregation problem with the guided filter. However, the guided filter does not make full use of the information of the depth image. Since depth images have quite sparse gradients, it inspires us to regularize the gradients for improving depth upscaling results. Statistics show a special property of depth images, that is, there is a non-ignorable part of pixels whose horizontal or vertical derivatives are equal to +/- 1. Based on this special property, we propose a low gradient regularization method which reduces the penalty for horizontal or vertical derivative +/- 1, and well describes the statistics of the depth image gradients. Then, we present a solution to the low gradient minimization problem based on threshold shrinkage. Finally, the proposed low gradient regularization is integrated with the guided filter into the depth image upsampling method. Experimental results demonstrate the effectiveness of our proposed approach both qualitatively and quantitatively compared with the state-of-the-art methods.