Properties of the Optimality Equation and Optimal Policies in Discrete Time Markov Decision Processes and Their Applications

摘要

This paper investigates properties of the optimaiity equation and optimal policies in discrete time Markov decision processes with expected discounted total rewards. Under conditions where the model is well defined and the optimaiity equation is true, it is shown that as a solution of the optimaiity equation, the solution called optimal value function is always the smallest one, and is also the unique one under another weak condition. Moreover, a structure of optimal policies is discussed. Finally, these properties are applied to state feedback control of discrete event systems with a numerical example.