High-Performance Data Stream Mining by Means of Embedding Hidden Markov Model into Reproducing Kernel Hibert Spaces

摘要

Hidden Markov models (HMMs) are a well-known probabilistic graphical model for time series of discrete, partially observable stochastic processes. We consider method to extend the application of HMMs to non-Gaussian continuous distributions by embedding the belief about the state into a reproducing kernel Hilbert space (RKHS). Corresponding regularization techniques are proposed to reduce tendency to overfitting and computational complexity of algorithm, specifically, Nystr?m subsampling for feature and kernel matrices and general regularization family. This method may be applied to various statistical inference and learning problems, including classification, clustering, prediction, identification, and as an online algorithm is may be used for dynamic data mining and data stream mining. We investigate, both theoretically and empirically, regularization and approximation bounds. Furthermore, we discuss applications of the method to real-world problems, comparing the approach to several state-of-the-art algorithms.