Relative-Entropy Minimization with Uncertain Constraints--Theory and Application to Spectrum Analysis.

摘要

The relative-entropy principle ('principle of minimum cross entropy') is a provably optimal information theoretic method for inferring a probability density from an initial ('prior') estimate together with constraint information that confines the density to a specified convex set. Typically the constraint information takes the form of linear equations that specify the expectation values of given functions. This paper discusses the effect of replacing such linear-equality constraints with quadratic constraints that require linear constraints to hold approximately, to within a specified error bound. The results are applied to the derivation of a new multisignal spectrum-analysis method that simultaneously estimates a number of power spectra given: (1) an initial estimate of each; (2) imprecise values of the autocorrelation function of their sum; (3) estimates of the error in measurement of the autocorrelation values. One application is to separate estimation of the spectra of a signal and independent additive noise, based on imprecise measurements of the autocorrelations of the signal plus noise. The new method is an extension of multisignal relative-entropy spectrum analysis (with exact auto-correlations). The two methods are compared, and connections with previous related work are indicated. Mathematical properties of the new method are discussed, and an illustrative numerical example is presented. Originator-supplied keywords include: Maximum entropy, cross entropy, Relative entropy, Information theory, Prior estimates, and Initial estimates.