Minimization of Variance on Controlled Markov Chain

摘要

We consider a variance criterion on a finitestage controlled Markov chain. The variance is a sample variance for the sequence of stage rewards (random variables) on the chain. Through a nonconventional dynamic programming approach,w e minimize the expected value o the variance over some large policy class. For computational simplicity,w e take the variance multiplied by the square of the total number of stages. Our invariant imbedding method expands the original state space by one dimension. We illustrate a two-state, two-action and two-stage model by stochastic decision tree-table method. This optimal solution is obtained by solving the backward recursive equation for the minimum value functions on expanded state spaces.