Recovery of a missing color component in stereo images (or helping NASA find little green Martians)

摘要

The current exploration of Mars by the National Aeronautics and Space Administration (NASA) has produced a lot of images of its surface. Two rovers, "Spirit" and "Opportunity", are each equipped with a pair of high-resolution cameras, called "PanCam". While most commercial cameras are sensitive to three spectral bands, typically red (R), green (G) and blue (B), the "PanCam" is sensitive to many more bands since it was designed to deliver additional information to geologists. This is achieved by means of a filter wheel in front of each camera lens. It turns out that slightly different filters are used in both cameras; while the left camera is equipped with red, green and blue filters, among others, the right camera does not have a green filter on its color wheel. Therefore, since the G component of the right image is missing, currently it is not possible to view a 3D image of Mars surface in color. In this paper, we develop a method to reconstruct one missing color component of an image given its remaining color components and all three components of the other image of a stereo pair. The method relies on disparity-compensated prediction. In the first step, a disparity field is estimated using the two available components (R and B). In the second step, the missing component is recovered using disparity-compensated prediction from the same component (G) in the other image of the stereo pair. In ground-truth experiments, we have obtained high PSNR values of the reconstruction error confirming efficacy of the approach. Similar reconstructions using images transmitted by the rovers yield comfortable 3D experience when viewing with shutter glasses.