Functional Approximation by Feed-Forward Networks: A Least-Squares Approach toGeneralisation

摘要

This paper considers a least-squares approach to function approximation andgeneralization. The particular problem addressed is one in which the training data is noiseless (perhaps specified by an assumed model or obtained during some calibration procedure) and the requirement is to define a mapping which approximates the data and which generalises to situations in which data samples are corrupted by noise. The least-squares approach produces a generaliser which is the vector of posterior probabilities and has the form of a Radial Basis Function network for a finite of training samples. The finite sample approximation is valid provided that the noise on the expected operating conditions is large compared to the sample spacing in the data space. In the other extreme of small noise perturbations, it is shown that better generalisation will occur if the training error criterion (the sum-square error on the training set) is modified by the addition of a specific regularisation term. This is illustrated by an approximator which has feed-forward architecture and applied to the problem of point-source location using the outputs of an array of receivers in the focal-plane of the lens.