Integrated Simulation-Based Methodologies for Planning and Estimation

摘要

Significant progress was made in a number of proposed research areas. The first major task in the proposal involved incorporating simulation-based optimization (and, in particular, ordinal optimization) into dynamic optimization problems. In support of this task, progress was made on new sampling methods for Markov Decision Processes (MDPs), a new time aggregation approach for MDPs, simulation-based methods for weighted cost-to-go MDPs, approaches to proving the exponential convergence rate of ordinal comparisons, approximate receding horizon approaches to MDPs and Markov games, and new classes of stochastic approximation algorithms. In support of the second major task that involved estimation and control algorithms for dynamic hierarchical and graphical models, a variety of algorithms and analytical tools were developed for models on graphs with loops that exploit embedded loop-free structure. These algorithms offer the potential of significantly enhanced solutions to a variety of optimization problems critical to the Air Force. Another major task in the proposal involved risk-sensitive estimation and control. In support of this task, a new filtering scheme for the risk-sensitive state estimation of partially observed Markov chains was introduced and analyzed.