Multi-Objective Markov Decision Processes for Data-Driven Decision Support

摘要

We present new methodology based on Multi-Objective MarkovDecision Processes for developing sequential decision supportsystems from data. Our approach uses sequential decision-makingdata to provide support that is useful to many differentdecision-makers, each with different, potentially time-varyingpreference. To accomplish this, we develop an extension offitted-$Q$ iteration for multiple objectives that computespolicies for all scalarization functions, i.e. preferencefunctions, simultaneously from continuous-state, finite-horizondata. We identify and address several conceptual andcomputational challenges along the way, and we introduce a newsolution concept that is appropriate when different actions havesimilar expected outcomes. Finally, we demonstrate anapplication of our method using data from the ClinicalAntipsychotic Trials of Intervention Effectiveness and show thatour approach offers decision-makers increased choice by a largerclass of optimal policies. color="gray">