Sensitivity Analysis for Stationary Probabilities of a Markov Chain

摘要

This paper considers the problem of evaluating the sensitivity of a steady state cost alpha (Theta) to underlying uncertainty in a parameter vector 0 governing the probabilistic dynamics of the system under consideration. We show that the gradient grad alpha (Theta) plays a fundamental role in the parametric statistical theory for Markov processes. We then survey numerical methods available for evaluating grad alpha (Theta) and introduce a new Monte Carlo estimator for grad alpha (Theta), which is applicable to Markov processes of substantial generality.