Analysis on the Weight initialization Problem in Fully-connected Multi-layer Perceptron Neural Network

摘要

In this paper, the weight initialization problem in fully connected multi-layer perceptron neural networks (MLPNN) is studied. A structure as 8-10-1 MLP neural network is taken as the experimental object. The training and predicting dataset selected in this paper is California housing price dataset. Eight special weight initialization methods are analyzed and five of them are tested by simply using the basic neural network with structure as 8-10-1 MLP to simulate one common neural network, visualize and compare the weight change process when using random weight initialization and its special weight initialization method. This paper implements the aim that identifying the influence of their special weight initialization method on their weight change process, focusing on what specific problems are solved by these special weight initialization methods to improve the convergence speed and accuracy and verifying the hypothesis that these specific weight initialization method will generate better result with high convergence speed and accuracy than random weight initialization.