Computational Comparison of Value Iteration Algorithms for Discounted Markov Decision Processes.

摘要

This note describes the results of a computational comparison of value iteration algorithms suggested for solving finite state discounted Markov decision processes. Such a process visits a set of states S = (1,2,...M). In Section two we describe the schemes examined and the various bounds that can be used for stopping them. Section three concentrates on one scheme that did well in the comparison - ordinary value iteration - and looks at various methods for eliminating non-optimal actions both permanently and temporarily.