Censoring Markov Chains and Stochastic Bounds

摘要

We show how to combine censoring technique for Markov chain and strong stochastic comparison to obtain bounds on rewards and the first passage time. We present the main ideas of the method, the algorithms and their proofs. We obtain a substantial reduction of the state space due to the censoring technique. We also present some numerical results to illustrate the effectiveness of the method.