Relation between weight initialization of neural networks and pruning algorithms: case study on Mackey-Glass time series

摘要

The implementation of weight initialization is directly related to the convergence of learning algorithms. We made a case study on the Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is the Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.