Theorem Connecting Adaptive Feed-Forward Layered Networks and Nonlinear Discriminant Analysis

摘要

A theorem which illustrates why a general adaptive feedforward layered network with linear output units can perform well as a pattern classification device is presented. The central result is that minimizing the error at the output of the network is equivalent to maximizing a particular norm, the network cost function, at the output of the hidden units. If the total covariance matrix is full rank and the targets are appropriately chosen, this cost function relates the inverse of the total covariance matrix and the weighted between class covariance matrix of the hidden unit patterns. In a linear network it is shown how the theorem can reproduce the result obtained by Gallinari et al., (1988) as a special case. Numerical simulations illustrate the theorem and show that alternative choices for the cost function at the hidden layer are not maximized, generally, in a nonlinear situation.