Applications of Numerical Methods for Stochastic Controlled Switching Diffusions with a Hidden Markov Chain: Case Studies on Distributed Power Management and Communication Resource Allocation

摘要

Recently, considerable attention has been drawn to stochastic controlled systems with hidden Markov chains. Much motivation stems from applications in distributed power management and platoon inter-vehicle distance maintenance, among others. The dynamic systems of interest are controlled diffusions with switching, known as switching diffusions. Different from the extensive studies contained in the aforementioned reference, the switching process in this paper is assumed to be a continuous-time Markov chain that is hidden. We can only observe the state of the Markov chain with additive noise. Mean-variance control problems were first considered in the Nobel prize winning paper of Markowitz. It was subsequently considered by a host of researchers. The recent advances in backward stochastic differential equations enable the treatment of the mean-variance controls in continuous time, which is otherwise impossible because of the so-called indefinite control weights; see Zhou and Li for the first paper in this direction and further details. Further work in conjunction with regime-switching models can be found in Zhou and Yin, among others.