Analysis of an Identification Algorithm Arising in the Adaptive Estimation of Markov Chains.

摘要

This reprint investigates an algorithm applied to the adaptive estimation of partially observed finitestate Markov chains. The algorithm utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. It is shown that the process driving the algorithm has a unique invariant measure for each fixed value of the parameter, and following the ordinary differential equation method for stochastic approximations, establish almost sure convergence of the parameter estimates to the solutions of an associated differential equation. The performance of the adaptive estimation scheme is analyzed by examining the induced controlled Markov process with respect to a long-run average cost criterion. (KR)