MW-OBS: An Improved Pruning Method for Topology Design of Neural Networks

摘要

Topology design of artificial neural networks (ANNs) is an important problem for large scale applications. This paper describes a new efficient pruning method, the multi-weight optimal brain surgeon (MW-OBS) method, to optimize neural network topologies. The advantages and disadvantages of the OBS and unit-OBS were analyzed to develop the method. Actually, optimized topologies are difficult to get within reasonable times for complex problems. Motivating by the mechanism of natural neurons, the MW-OBS method balances the accuracy and the time complexity to achieve better neural network performance. The method will delete multiple connections among neurons according to the second derivative of the error function, so the arithmetic converges rapidly while the accuracy of the neural network remains high. The stability and generalization ability of the method are illustrated in a Java program. The results show that the MW-OBS method has the same accuracy as OBS, but time is similar to that of unit-OBS. Therefore, the MW-OBS method can be used to efficiently optimize structures of neural networks for large scale applications.