A note on deterministic approximation of discounted Markov decision processes

摘要

We study the approximation of a small-noise Markov decision process x(t) = F(x(t-1), a(t), xi(t)(epsilon)), t = 1, 2, ... by means of its deterministic counterpart: (x) over tilde (t) = F((x) over tilde (t-1), a(t), s(0)), t = 1, 2, ... where s(0) is a fixed point of the disturbance metric space (S, r). The total discounted cost is used as a criterion of optimality. Supposing that delta(epsilon) := Er(xi(1)(epsilon), s(0)) -> 0 as epsilon -> 0, we prove the convergence of optimal policies, estimate the rate of convergence of the optimal costs and give an upper bound (depending on delta(epsilon)) for the stability index. which measures the excess of the cost due to a replacement of the optimal policy by its deterministic approximation.