A Lyapunov Shortest-Path Characterization for Markov Decision Processes

摘要

In this paper we introduce a modeling paradigm for developing decision process representation for shortest-path problems. Whereas, in previous work attention was restricted to tracking the net using Bellman's equation as a utility function, this work uses a Lyapunov-like function. In this sense, we are changing the traditional cost function by a trajectory-tracking function which is also an optimal cost-to-target function for tracking the net. The main point of the Markov decision process is its ability to represent the system-dynamic and trajectory-dynamic properties of a decision process. Within the system-dynamic properties framework we prove new notions of equilibrium and stability. In the trajectory-dynamic properties framework, we optimize the value of the trajectory-function used for path planning via a Lyapunov-like function, obtaining as a result new characterizations for final decision points (optimum points) and stability. Moreover, we show that the system-dynamic and Lyapunov trajectory-dynamic properties of equilibrium, stability and final decision points (optimum points) meet under certain restrictions.