Convergence of online gradient methods for Pi-Sigma neural network with a penalty term

摘要

Pi-Sigma network is a class of feed-forward neural networks with product units in the output layer. An online gradient method with penalty which is proportional to the norm of the weights is presented and the convergence results are discussed for a Pi-Sigma neural network. The weights are proved to be bounded in the training process, thus the conditions that are required for convergence of online gradient method in literature are simplified.