Illumination-invariant image retrieval and video segmentation

摘要

Images or videos may be imaged under different illuminants than models in an image or video proxy database. Changing illumination color in particular may confound recognition algorithms based on color histograms or video segmentation routines based on these. Here we show that a very simple method of discounting illumination changes is adequate for both image retrieval and video segmentation tasks. We develop a feature vector of only 36 values that can also be used for both these objectives as well as for retrieval of video proxy images from a database. The new image metric is based on a color-channel-normalization step, followed by reduction of dimensionality by going to a chromaticity space. Treating chromaticity histograms as images, we perform an effective low-pass filtering of the histogram by first reducing its resolution via a wavelet-based compression and then by a DCT transformation followed by zonal coding. We show that the color constancy step - color band normalization - can be carried out in the compressed domain for images that are stored in compressed form, and that only a small amount of image information need be decompressed in order to calculate the new metric. The new method performs better than previous methods tested for image or texture recognition and operates entirely in the compressed domain, on feature vectors. Apart from achieving illumination invariance for video segmentation, so that, e.g. an actor stepping out of a shadow does not trigger the declaration of a false cut, the metric reduces all videos to a uniform scale. Thus thresholds can be developed for a training set of videos and applied to any new video, including streaming video, for segmentation as a one-pass operation.