Fast and Stable Learning Utilizing Singular Regions of Multilayer Perceptron

摘要

In the parameter space of MLP(J), multilayer perceptron with J hidden units, there exist flat areas called singular regions created by applying reducibility mappings to the optimal solution of MLP(7 - 1). Since such singular regions cause serious stagnation of learning, a learning method to avoid singular regions has been desired. However, such avoiding does not guarantee the quality of the final solutions. This paper proposes a new learning method which does not avoid but makes good use of singular regions to stably and successively find excellent solutions commensurate with MLP(J). The proposed method worked well in our experiments using artificial and real data sets.