Comparing Adaptive and Non-Adaptive Connection Pruning With Pure Early Stopping

摘要

Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization, as is shown in this empirical study. However, an open problem in the puning methods known today (OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This work presents a pruning method lprune that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requries no algorithm parameter adjustment by th euser. Results of statistical given, based on extensive experimentation with 14 different problems. The results indicate that training with pruning without pruning. Furthermore, lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required.