Multiple Time Scale Decomposition of Discrete Time Markov Chains

摘要

The multiple time scale decomposition of discrete time, finite state Markov chains is addressed. In previous works, the behavior of a continuous time Markov chain is approximated using a fast time scale, epsilon-independent, continuous time process, and a reduced order perturbed process. The procedure can then be iterated to obtain a complete multiple time scale decomposition. In the discrete time case presented in this paper, the basic approximation has a 'hybrid' form. In this form, the fast time scale behavior is approximated using an epsilon-independent, discrete time Markov chain, and the slow behavior is captured by a perturbed, continuous time process. Further time scale decomposition then involves the continuous time procedure. This extension to discrete time chains bridges previous multiple time scale decomposition results, which have dealt exclusively with either continuous time or discrete time processes, and provides a uniform framework for the analysis of both types of systems. Keywords: Markov process, Discrete time, Aggregation, Perturbation theory, Multiple time scales, Reprints. (KR)