Canonical Dynamic Programming Equations subject to Ambiguity ?

摘要

This paper studies the infinite horizon average cost Markov control model subject to ambiguity on the controlled process conditional distribution. The stochastic control problem is formulated as a minimax optimization in which, (i) the existence of optimal policies is established through a pair of canonical dynamic programming equations derived for Borel state and action spaces, and (ii) the controlled process maximizing conditional distribution is characterized through a water-filling solution derived for finite state and action spaces. To obtain average cost optimal policies numerically a policy iteration algorithm is also developed. Finally, as an application of the proposed canonical dynamic programming equations, an example is provided.