THE COMBINED STATISTICAL STEPWISE AND ITERATIVE NEURAL NETWORK PRUNING ALGORITHM

摘要

In this paper, we present a new pruning algorithm formed by combining the Statistical Stepwise Method (SSM) [1] with the Iterative Pruning (IP) [4] algorithms. This proposed algorithm (SSIP) is used to simultaneously remove unnecessary neurons or weight connections from a given feed-forward neural network (NN) in order to "optimize" its structure. Some modifications to the previous pruning algorithms published in [1] and [4] are also reported. Two versions of the combined SSIP are considered: in the first version. SSIP1. the modified IP is first applied to the given neural network in order to prune insignificant units, and then the modified SSM is applied to the pruned network to remove unnecessary links. In the second version. SSIP2, the above procedure is applied to each layer in turn, working from the input layer to the output layer. The performances of the algorithms are compared using two real world applications, brain disease detection and texture classification, and the superiority of the SSIP pruning algorithm is demonstrated. This new algorithm can eliminate approximately 59% of the links in the initial oversized network in order to improve performance by approximately -39% and - 26% for the sensitivity of learning, and generalization, respectively.