Smart Vision Image Processing: A Priori MaxEnt H(S) ica vs. A Posteriori MaxEnt H(V) ICA

摘要

Two mirror symmetric versions of the maximum entropy (MaxEnt) methodology are introduced and compared: (1) A posteriori MaxEnt Independent Component Analysis (ICA) H(V) was proposed by Bell, Sejnowski, Amari, Oja (BSAO) (early by Jutten & Herault, Comon and Cardoso (JHCC) in France). It wishes to factorize the unknown joint probability density function (pdf) using the multi-channel data vector X(x,y), coined by Comon as ICA by means of a post processing algorithm of output V(x,y) = σ ([W]X(x,y)) at each pixel location (x,y) via the unsupervised learning algorithm, (partial deriv)[W]/(partial deriv)t = <(partial deriv)H(V)/(partial deriv)[W]>. The pixel-ensemble average denoted by the angular brackets is necessary to estimate the underlying pdf, which is complete but over-ambitious for a simple classification task. The latter task may be referred to as "independent class analysis (ica)" in lower case, in contrast to "ICA" in upper case intended for obtaining the detailed pdf description. (2) A priori MaxEnt H(S) for ica of S could be a complimentary first step to ICA of their underlying pdf, namely an areaintegral of pdf within class boundaries. Since ica is a single realization of the ensemble, we can derive directly from Gibb's statistics mechanics of independent classes of irradiation sources S by the a priori MaxEnt H(S), which would be equally flat if each were not constrained by measurements by means of the Lagrangian multipliers of force vector λ_i and energy scalar (λ_o -1) per pixel: H(S_j) = -Σ_(j=1)~NS_j ln(S_j) - Σ_(i=1)~NΣ_(j=1)~Nλ_i(A_(ij)S_j -X_i~((known))) -(λ_0 - 1)(Σ_(j=1)~NS_j - 1) Geometric optics speaking, the long distance propagation by the speed of the light insures the linear and instantaneous characteristic and the line-of-sight validity of within-pixel mixture of the ground irradiant sources in the remote sensing using optical spectra. This fact should be independent of any algorithm. Without the pixel ensemble average computational limitation, however, the a priori MaxEnt H(S) ica can handle large dimensional and large size imageries such as hyperspectral image data basis (200 spectral channels; 200 * 200 pixels each) by the pixel-by-pixel divide-and-conquer strategy. This is possible because the ground irradiation source S(x_o,y_o) per pixel contributes locally to the corresponding image pixel X(x_o,y_o) = [A]S(x_o,y_o), i.e. the spectral energy collected within the ground "footprint of the pixel (x_o,y_o)" will not be mixed with other neighborhood sources due to the optics imaging lens. Although this is basically true for any processing algorithm, for the BSAO algorithm attempting comprehensively the underlying pdf ICA, all pixel ensemble [W]X(x,y) are utilized in sweeping through a randomly permuted batch mode that makes it limited to a lower dimension image-size (upto 7 spectral channels of 200 * 200 pixels each). Being less comprehensive, the a priori MaxEnt H(S) ica can compute S-decomposition, namely the area-integral of pdf under the class boundaries, pixel by pixel, in real time. Furthermore, as a byproduct, the ab initio derivations of sigmoid threshold logic S = σ(λ[A]) and Hebbian learning rule ΔA_(ij) = λ_iS_j makes one wonder for compression efficiency reasons why such a general communication of data X = [A]S by means of linear independent classes leads to the sigmoid transfer logic under the constraint of unit class decomposition Σ_jS_j = 1. We conclude in passing the Lagrangian Constraint Neural Network (LCNN) is developed since 1997 and that allow nonlinear data generalization and conjugate gradient ascents of mirror symmetric MaxEnt's.