Estimation and control of jump stochastic systems.

摘要

This thesis deals with the estimation and control of jump stochastic systems as a result of the novel proposition of a framework based on switching hidden modes to enhance the treatment of uncertainty within the process control field. This thesis begins with the realistic modeling of non-stationary disturbance signals typically witnessed in process industries. Such disturbances are characterized by probabilistic switches between distinct regimes. To capture the effect of such multiple modes, a Hidden Markov Model (HMM) framework is employed. The main disturbance patterns of concern considered in this thesis include intermittent drifts and abrupt jumps. The main idea is to superimpose a Markov chain, whose parameters govern the latent temporal regime transitions on top of a discrete-time equation that governs the underlying system dynamics to generate a (concatenated) Markov Jump System (MJS). Several examples are given in the thesis which demonstrate the usefulness of the HMM framework in the context of describing an array of disturbance patterns, providing integral action and the provision for model-based fault detection. The main contribution is practical in that with the adoption of a more sophisticated disturbance model, and the consequential use of an existing state estimator suited for MJS's, superior tracking performance is possible. Enhanced closed-loop control, without having to re-design the controller (which remains as MPC for this part of the thesis), is the result.;Having dealt with the issue of modeling, the thesis then proceeds to the optimal control of stochastic systems, including the aforementioned jump systems. The control framework employed is Approximate Dynamic Programming (ADP) based on a "post-decision state", as opposed to the more common "pre-decision" state. ADP is a promising framework for systematically and practically obtaining good closed-loop policies for multi-stage, stochastic control problems. The main advantage of ADP over MPC is the former's ability to account for uncertainty in a systematic fashion. Furthermore, for ADP, the bulk of the computation burden is shifted off-line (during a process termed Value Iteration (VI), via which "value-functions" are computed). Online computations are oftentimes swifter, since the solution to a single-stage optimization problem (as opposed to a multi-stage one, in the case of MPC) is required. The contributions in this part of the thesis are multi-fold.;Most previous works on ADP, as applied in the context of process control, have focused on solving deterministic problems. For stochastic control problems, the pre-decision ADP formulation involves a computationally cumbersome optimization over an expected quantity during on-line and off-line calculations. Through the use of the post-decision state, the typically non-commutative optimization and expectation operations are interchanged to yield an equivalent problem. The post-decision stage formulation also allows the effcient use of off-the-shelf optimization solvers which form the cornerstone of MPC technology. A further benefit is that off-line VI calculations may be run in parallel. This thesis extends previous ADP methodologies (involving simulations to uncover relevant parts, over which VI is performed, of the state-space and function approximation) to the post-decision analogue.;The proposed post-decision-state-based ADP approach is demonstrated on but not limited to the control of stochastic jump systems. Several examples are used to demonstrate the algorithm and to highlight the inadequacy of MPC (and/or other popular control methodologies) in providing good closed-loop control due to its ad-hoc treatment of uncertainty. These examples include a dual control problem (of an integrator with an unknown gain) and a case study highlighting the importance of considering the oftentimes over-looked interaction between state-estimation and control. Another example is one of a constrained double integrator, where MPC leads to frequent violations of the constraints due to unbounded prediction errors. The last pertains to a bioreactor chemostat where the aim of maximizing productivity whilst ensuring high conversion results in operations at the constraint boundary. The proposed ADP solution is able to automatically "back-off" from the constraint boundary in the face of disturbances. (Abstract shortened by UMI.)