Probabilistic planning models Markov Decision Processes(MDPs) and Partially Observable Markov Decision Processes (POMDPs) -- are models where autonomous agents operate and make sequential decisions under uncertainty environment in successive episodes. To adapt to different scenario and objectives, many criteria and variants for MDPs and POMDPs have been proposed. The Risk-Sensitive criterion (RS-criterion) is one criterion that maximize the probability that the accumulated cost of the agent execution is less than a predefined threshold, so that the agent can avoid the situation that an exorbitantly high accumulated cost is encountered as much as possible.;Risk-Sensitive MDPs (RS-MDPs) and Risk-Sensitive POMDPs (RSPOMDPs) are risk-sensitive models that combine the RS-criterion with MDPs and POMDPs, respectively. I hypothesize that one can design novel algorithms which are specific for RS-MDPs and RS-POMDPs by applying insights gained from analyzing properties and structures of risk-sensitive models. The key observation about risk-sensitive models is that the original formalizations can be transformed to new formalizations by maintaining the consumed cost so far and considering it for decisions as well.;To validate my hypothesis, (1) I formally define the RS-MDP model and distinguish the models under different assumptions about cost. For each assumption and model; I introduce new algorithms and discuss related properties. (2) I formally propose the RS-POMDP model and discuss some deficiencies of several existing regular POMDP models. I introduce procedures to complement several existing algorithms and also provide new algorithms for RS-POMDPs. (3) I also provide theoretical properties of the risk-sensitive models as well as empirical evaluations of the new algorithms on randomly generated problems, the NAVIGATION domain from the ICAPS International Probabilistic Planning Competition, and a taxi domain generated with real-world data. For both RS-MDPs and RS-POMPs, the experiments show my algorithms are more efficient than existing algorithms, which validate my hypothesis.
展开▼
