A BOUNDARY NOISE REMOVAL AND HOLE FILLING METHOD FOR VIRTUAL VIEWPOINT IMAGE GENERATION

摘要

The present invention relates to a boundary noise removal and hole filling method for virtual viewpoint image generation capable of, when generating a virtual viewpoint depth image from a reference viewpoint depth image, expanding boundary noise presenting in the virtual viewpoint depth image to a hole region, respectively obtaining pixel values in the hole region by a spiral weighted average method and a gradient searching method, combining filling regions using a weighted value, and filling the hole region, the boundary noise removal and hole filling method comprising the steps of: (a) generating a virtual viewpoint image from the reference viewpoint depth image by a 3D warping method; (b) detecting a boundary noise region from a region adjacent to a hole and expanding pixels in the boundary noise region to the hole region; (c) filling each hole in the hole region with pixels by the spiral weighted average method to obtain a first filling region; (e) filling each hole in the hole region with pixels by the gradient searching method to obtain a second filling region; (f) combining the first and second filling regions using the weighted value to generate a final filling region; and (g) filling the hole region of the virtual viewpoint image with the final filling region to obtain a final virtual viewpoint image. Thus, the spiral weighted average method using the minimum object information by using depth information but obtaining a smeared image, and the gradient searching method preserving fine details by using the gradient of an image can be used to supplement the demerits of the two methods and apply the merits thereof at the same time, thereby obtaining a natural and good-quality image. [Reference numerals] (AA) Start; (BB) End; (S10) Generate a virtual viewpoint depth image; (S20) Detect a boundary noise region to expand to a hole region; (S30) Obtain a first filling region by a spiral weighted average method; (S40) Obtain a second filling region by a gradient searching method; (S50) Combine the first and second filling regions using a weighted value to generate a final filling region; (S60) Fill the hole region with the final filling region to obtain a final virtual viewpoint image