Superpixel-based color-depth restoration and dynamic environment modeling for Kinect-assisted image-based rendering systems

摘要

Depth information is an important ingredient in many multiview applications including image-based rendering (IBR). With the advent of electronics, low-cost and high-speed depth cameras, such as the Microsoft Kinect, are getting increasingly popular. In this paper, we propose a superpixel-based joint color-depth restoration approach for Kinect depth camera and study its application to view synthesis in IBR systems. Thus, an edge-based matching method is proposed to reduce the color-depth registration errors. Then the Kinect depth map is restored based on probabilistic color-depth superpixels, probabilistic local polynomial regression and joint color-depth matting. The proposed restoration algorithm does not only inpaint the missing data, but also correct and refine the depth map to provide better color-depth consistency. Last but not the least, a dynamic background modeling scheme is proposed to address the disocclusion problem in the view synthesis for dynamic environment. The experimental results show the effectiveness of the proposed algorithm and system.