Small Sample Behavior of Multi-Layer Feedforward Network Classifiers: Theoretical and Practical Aspects

摘要

Multi-layer feedforward network classifiers have become very popular tools for pattern recognition during the last decade. In the first part of the thesis, a number of theoretical issues on this topic are investigated. A review of the literature on small sample problems in pattern recognition is presented, followed by a discussion of an elegant mathematical framework to study small sample problems. This framework, known as the Vapnik-Chervonenkis theory, is applied to the class of multi-layer feedforward network classifiers in order to characterize their small sample properties. Moreover, the influence of the training procedure on the small sample behavior of the multi-layer network classifier is investigated. The second part of the thesis is concerned with a number of practical aspects of training a multi-layer feedforward network classifier. The problem of the stopping criterion of an iterative learning procedure is studied. Some issues on the learning speed of a class of iterative learning procedures are investigated.