Transmitting texture and depth images of captured camera view(s) of a 3D scene enables a receiver to synthesize novel virtual viewpoint images via Depth-Image-Based Rendering (DIBR). However, a DIBR-synthesized image often contains disocclusion holes, which are spatial regions in the virtual view image that were occluded by foreground objects in the captured camera view(s). In this paper, we propose to complete these disocclusion holes by exploiting the self-similarity characteristic of natural images via nonlocal template-matching (TM). Specifically, we first define self-similarity as nonlocal recurrences of pixel patches within the same image across different scales--one characterization of self-similarity in a given image is the scale range in which these patch recurrences take place. Then, at encoder we segment an image into multiple depth layers using available per-pixel depth values, and characterize self-similarity in each layer with a scale range; scale ranges for all layers are transmitted as side information to the decoder. At decoder, disocclusion holes are completed via TM on a per-layer basis by searching for similar patches within the designated scale range. Experimental results show that our method improves the quality of rendered images over previous disocclusion hole-filling algorithms by up to 3.9dB in PSNR.ud
展开▼
机译：传输3D场景捕获的相机视图的纹理和深度图像使接收器可以通过基于深度图像的渲染（DIBR）合成新颖的虚拟视点图像。然而，DIBR合成的图像通常包含遮挡孔，该遮挡孔是虚拟视图图像中的空间区域，这些空间区域被捕获的摄像机视图中的前景对象遮挡。在本文中，我们建议通过非本地模板匹配（TM）利用自然图像的自相似性特征来完成这些遮挡孔。具体而言，我们首先将自相似性定义为同一图像中跨不同尺度的像素斑块的非局部重复发生-给定图像中自相似性的一个特征是发生这些斑块重复发生的尺度范围。然后，在编码器中，我们使用可用的每像素深度值将图像分割为多个深度层，并使用比例尺范围描述每个层中的自相似性；所有层的比例范围作为辅助信息传输到解码器。在解码器上，通过搜索指定比例范围内的相似色块，通过TM在每层的基础上完成遮挡孔。实验结果表明，与以前的消隐孔填充算法相比，我们的方法将渲染图像的质量提高了3.9dB（PSNR）。 ud
展开▼
