Invariant Measure Approach to the Convergence of Stochastic Approximations with State Dependent Noise

摘要

A new method is presented for quickly getting the ODE (ordinary differential equation) associated with the asymptotic properties of a stochastic approximation. The method basically requires that the functions be Markov with a 'Feller' transition function, but little else. No mixing condition is required, nor the construction of averaged test functions, and f(.,.) need not be continuous. For the class of sequences treated, the conditions seem easier to verify than for other methods. There are extensions to the non-Markov case. Two examples illustrate the power and ease of use of the approach. Aside from the advantages of the method in treating standard problems, it is useful for handling the type of iterative algorithms which arise in adaptive communication theory, where the dynamics are often discontinuous and the 'noise' is often state dependent due to the effects of feedback.