A stochastic-approximation-based analysis of some dynamic neural network learning schemes

摘要

In this paper, discrete-tune stochastic models are presented for characterizing weight adjustment procedures in dynamic artificial neural networks. A stochastic approximation approach is employed for the analysis of these neural network learning schemes. Among the techniques usually employed in stochastic approximation theory, the ordinary differential equation (ODE) method is selected here. Since a continuous-time neural network dynamic model is considered, the ODE method shows to be suitable for analysis purposes providing a unified framework together with such neural model. In some situations, the approach leads to a set of linearly implicit differential equations. The analysis of these equations and the relationship with the original stochastic discrete-time formulations are considered.