Effect of the feature vector size on the generalization error: the case of MLPNN and RBFNN classifiers

摘要

In pattern recognition literature, it is well known that a finite number of training samples cause practical difficulties in designing a classifier. Moreover, the generalization error of the classifier tends to increase as the number of features gets large. We study the generalization error of several classifiers (MLPNN, RBFNN, K NN) in high dimensional spaces, under a practical condition: the ratio of the training sample to the dimensionality is small. Experimental results show that the generalization error of neuronal classifiers decreases as a function of dimensionality while it increases for statistical classifiers.