On the role of mismatch in rate distortion theory

摘要

Using a codebook C, a source sequence is described by the codewordthat is closest to it according to the distortion measured₀(x,xˆ₀). Based on this description, thesource sequence is reconstructed to minimize the distortion measured byd₁(x,xˆ₁), where in generald₁(x,xˆ₁)≠d₀(x,xˆ₀). We study the minimum resulting d₁(x,xˆ₁)-distortion between the reconstructed sequence and the sourcesequence as we optimize over the codebook subject to a rate constraint.Using a random coding argument we derive an upper bound on the resultingdistortion. Applying this bound to blocks of source symbols we constructa sequence of bounds which are shown to converge to the least distortionachievable in this setup. This solves the rate distortion dual of anopen problem related to the capacity of channels with a given decodingrule-the mismatch capacity. Addressing a different kind of mismatch, wealso study the mean squared error description of non-Gaussian sourceswith Gaussian codebooks. It is shown that the use of a Gaussian codebookto compress any ergodic source results in an average distortion whichdepends on the source via its second moment only. The source with agiven second moment that is most difficult to describe is the memorylesszero-mean Gaussian source, and it is best described using a Gaussiancodebook. Once a Gaussian codebook is used, we show that all sources ofa given second moment become equally hard to describe