High-Quality Multi-View Image Extraction from a Light Field Camera Considering Its Physical Pixel Arrangement

摘要

We propose a method for extracting multi-view images from a light field (plenoptic) camera that accurately handles the physical pixel arrangement of this camera. We use a Lytro Illum camera to obtain 4D light field data (a set of multi-viewpoint images) through a micro-lens array. The light field data are multiplexed on a single image sensor, and thus, the data is first demultiplexed into a set of multi-viewpoint (sub-aperture) images. However, the demultiplexing process usually includes interpolation of the original data such as demosaicing for a color filter array and pixel resampling for the hexagonal pixel arrangement of the original sub-aperture images. If this interpolation is performed, some information is added or lost to/from the original data. In contrast, we preserve the original data as faithfully as possible, and use them directly for the super resolution reconstruction, where the super-resolved image and the corresponding depth map are alternatively refined. We experimentally demonstrate the effectiveness of our method in resolution enhancement through comparisons with Light Field Toolbox and Lytro Desktop Application. Moreover, we also mention another type of light field cameras, a Raytrix camera, and describe how it can be handled to extract high-quality multi-view images.