Dynamic Bayesian networks for online stochastic modeling.

摘要

This dissertation introduces a novel learning methodology for a Dynamic Bayesian Network (DBN) and its applications to stochastic modeling of time-varying dynamics and nonstationary statistics. The parameters of the DBN are the probabilities of transition between discrete states. We express the transition probabilities as a linear combination of average likelihoods and update the average likelihoods recursively based on observation sets. An adaptive sliding window allows flexible online learning in which the length of the data record is determined based on data spread. Using stochastic convergence and dynamic stability theorems, we prove the asymptotic convergence of the proposed algorithm. We also use stochastic Lyapunov stability theory to establish the convergence of a generic time-varying DBN model.; Next, we develop a neural network based Active Queue Management (AQM) for Transmission Control Protocol (TCP) network, in which our DBN parameter estimation is embedded to model its input/output dynamics. The neural AQM has three neural controls which are independently trained under different TCP topologies. Simulation using OPNET modeler(c) shows that the proposed AQM scheme performs better than traditional AQM.; Next, the online DBN algorithm is used to recursively estimate time-varying parameters of the continuous Birth-Death process. The birth and death rates, as the parameters of the process, are dynamically modified via the proposed DBN technique to adaptively reflect its nonstationary statistics. The algorithm is applied to estimating road traffic.; A final application of the DBN approach includes discrete and non-Gaussian probability density estimation. We define the posterior state probability density from a simple DBN structure in which the transition probability is expressed as a linear combination of kernel functions. We determine the optimal estimate of the state probability by a Maximum Likelihood (ML) routine. An alternative framework applies our DBN learning algorithm to estimate the transition stochastic matrix in an online procedure. By estimating the stochastic matrix, we obtain the corresponding posterior probability density. Simulation examples demonstrate the outstanding performance of our proposed approaches, particularly for a nonstationary environment.