Shift-Function Approach for Markov Decision Processes with Unbounded Returns

摘要

The basic Markov decision model is described and the notation that is used throughout the report is established. The notation incorporates the discount factor in the transition probabilities and allows for defective transition probabilities. Two special cases of the basic decision model are studied, both of which abstract some of the properties commonly found in stochastic service and storage systems. A third model is investigated which also allows unbounded return functions, but places fewer restrictions on the return functions and transition probabilities, and is thus applicable to a wider class of Markov decision processes. Examples are given to support the assertion that the conditions of this model are satisfied by nearly all queuing-control models considered in the literature. An inventory control model is also presented.