Partially Observable Markov Decision Processes with an Average Cost Criterion.

摘要

We consider partially observable Markov decision processes with finite or countable (core) state and observation spaces and finite control space. Following a standard approach, an equivalent completely observed problem is formulated, with the same finite control space but with an uncountable state space, namely the space of probability distributions on the original core state space. It is observed that some characteristics induced in the original problem due to the finiteness, or countability, of the spaces involved are reflected onto the equivalent problem. Sufficient conditions are then derived for a bounded solution to the average cost optimality equation to exist. We illustrate these results in the context of machine replacement problems. Structural properties for average cost policies are obtained for a two state replacement problem, similar to available results for discount optimal policies. The set of assumptions used seems to be significantly less restrictive than others currently available. Keywords: Reprints. (kr)