Comparison of Depth Image-Based Rendering and Image Domain Warping in 3D Video Coding.

摘要

3D became successful in the movie theaters but failed to become mainstream for home use. The inconvenience of wearing glasses is arguably the reason and researchers have been investigating solutions for glasses-free 3D displays. Today, the most promising solution is the autostereoscopic display, which require many views of the same scene to be displayed simultaneously for a comfortable viewing experience. However, coding 3D video (3DV) with too many views is impractical with current networks and the 3DV coding standard, H.264/MVC (MVC), as the necessary bitrate is linearly proportional to the number of coded views. Instead, only a sparse set of anchor views can be compressed with some supplementary data and the remaining can be synthesized at the decoder.;In this dissertation, we compare two very popular view synthesis methods, Depth Image-Based Rendering (DIBR) and Image Domain Warping (IDW), in terms of their coding efficiency and complexity. First, we establish a common formulation that allows us to compare DIBR and IDW and their associated 3DV representations mathematically.;Then we provide the details of a fast DIBR-based view synthesis method and its implementation on GPU. We show that it can synthesize views with good objective quality and can provide inter-view consistency with almost constant time complexity in terms of the number of synthesized views.;Moreover, we present a new coding tool, "Depth-based Prediction Mode" (DBPM), and incorporate it into the coding loop of MVC. Using DBPM, we realize a novel MVD codec and we show that view synthesis can also be used for better prediction of the anchor views. DBPM uses the supplementary depth data and DIBR to achieve up to 9.2%, 9.9% and 6.7% bitrate savings over MVC for coding MVD data, depth maps and multiview videos, respectively.;Finally, we establish a codec framework based on the next generation candidate 3DV coding standard (3D-AVC), which has prediction tools similar to the DBPM already incorporated, and show that both DIBR and IDW can be used in this framework without any syntax changes to the standard. Using this framework we show that IDW achieves better coding performance than DIBR with average bitrate savings of 12.8% for anchor views and 1.5% for the synthesized views with significantly lower computational complexity. Finally, we provide an analysis on the effect of camera noise on measuring the quality of synthesized views with DIBR and IDW and show that camera noise produce a bias towards better measurements for DIBR. Recalculating the bitrate savings on sequences without camera noise shows that IDW can actually achieve average bitrate savings of 8.8% in the synthesized views instead of 1.5%.