In this contributon we evaluate on-line and off-line techniques to train a singleudhidden layer neural network classifier with logistic hidden and softmax output transferudfunctions on a multispectral pixel-by-pixel classification problem. In contrast toudcurrent practice a multiple class cross-entropy error function has been chosen as theudfunction to be minimized. The non-linear diffierential equations cannot be solved inudclosed form. To solve for a set of locally minimizing parameters we use the gradientuddescent technique for parameter updating based upon the backpropagation techniqueudfor evaluating the partial derivatives of the error function with respect to theudparameter weights. Empirical evidence shows that on-line and epoch-based gradientuddescent backpropagation fail to converge within 100,000 iterations, due to the fixedudstep size. Batch gradient descent backpropagation training is superior in terms ofudlearning speed and convergence behaviour. Stochastic epoch-based training tends toudbe slightly more effective than on-line and batch training in terms of generalizationudperformance, especially when the number of training examples is larger. Moreover, itudis less prone to fall into local minima than on-line and batch modes of operation. (authors' abstract)
展开▼
