Robust topological policy iteration for infinite horizon bounded Markov Decision Processes

摘要

Markov Decision Processes (MDPS) are commonly used to solve sequential decision problems. A less restrictive model is the Bounded-parameter MDP (BMDP) that allows: (i) the transition function to be expressed in terms of probability intervals and (ii) reasoning about a robust solution, i.e., the best solution under the worst model. In this paper, we propose the Robust Topological Policy Iteration (RTPI) algorithm which is a new policy iteration algorithm for infinite horizon BMDPs based on a partition of the state space. The empirical results show that the more structured the domain, the better is the performance of RTPI. (C) 2018 Elsevier Inc. All rights reserved.