Convergence and objective functions of noise-injected multilayer perceptrons with hidden multipliers

摘要

Artificial neural networks (ANNs) are known to be sensitive to the initial setting of parameters and the network architecture, such as the number of hidden nodes in multilayer perceptron (MLP). In this paper, we focus on a network structure which can help to find the proper number of hidden nodes in MLP. In this structure, so called Multilayer Perceptrons with Hidden Multipliers (MLPHM), each of the hidden nodes is associated with a tunable "gate" multiplier. With a specific regularization term, each gate tends to be opened or closed completely at the end of the training, and finally a pruned network is obtained. To study the fault tolerance and to improve the generalization of MLPHM, a noise-injected training scheme is proposed, with both multiplicative noise and additive noise taken into consideration. The objective functions and convergence theorems of the noise-injected training algorithms are obtained, and the latter have been verified by simulations. Applications to several UCI datasets have demonstrated that the proposed algorithms have efficient pruning ability and superior generalization ability. (c) 2020 Elsevier B.V. All rights reserved.