Efficient Synthetic Refocusing Method from Multiple Coded Aperture Images for 3D User Interaction

摘要

In this paper, we propose an efficient synthetic refocusing method from multiple coded aperture images for 3D user interaction. The proposed method is applied to a flat panel display with a sensor panel which forms lens-less multi-view cameras. To capture the scene in front of the display, the modified uniformly redundant arrays (MURA) patterns are displayed on the LCD screen without the backlight. Through the imaging patterns on the LCD screen, MURA coded images are captured in the sensor panel. Instead of decoding all coded images to synthetically generate a refocused image, the proposed method only decodes one coded image corresponding to the refocusing image at a certain distance after circularly shifting and averaging all coded images. Further, based on the proposed refocusing method, the depth of an object in front of the display is estimated by finding the most focused image for each pixel through a stack of the refocused images at different depth levels. Experimental results show that the proposed method captures an object in front of the display, generates refocused images at different depth levels, and accurately determines the depth of an object including real human hands near the display.