Regularized deterministic annealing EM for hidden Markov models.

摘要

In exploratory scientific research it is common to find oneself in situations in which the observational data greatly outpaces the available explanatory theory. In such a scenarios simple statistical models form a valuable stage in the research cycle, providing insight into the data that in turn provides inspiration for development of further scientific theory, to be confirmed by experiment or additional observation. The challenge in such a context is to perform optimization of the model while introducing as little bias as possible, as we do not want to eliminate the unexpected results that are often the most valuable.; Since many physical systems are both time dependent and undergo distinct state changes, hidden Markov models (HMMs) are a good candidate for analysis of many scientific data sets. Unfortunately, optimization of HMMs suffers from the well-known problem of local maxima. Applications of HMMs to date have focused primarily on areas, such as speech recognition and protein sequence analysis, where a priori knowledge can be used to constrain the problem and thereby address local maxima. In this work we tackle the problem of optimizing HMMs in situations where such a priori information is not available.; We present evidence that the problem is not addressed by existing methodologies, and devise an alternative approach based on applying the deterministic annealing expectation-maximization (EM) algorithm to the unsupervised train ing of HMMs. Our testing of the deterministic annealing method reveals that it suffers from a certain class of systemic local maxima which are characterized by the presence of redundant states with identical output distributions. We address this problem by designing statistical priors that bias the solution away from these maxima; because these priors address only systemic maxima we can be confident that they do not interfere with the ability of the model to learn any specific property of the observation data.; We present evaluations of the performance of the method on both synthetic and field instrument data, demonstrating the superior ability of the method to avoid local maxima as compared to the standard and deterministic annealing EM algorithms. In addition, we present mathematical analysis showing that for common data types there is an exponential lower bound on the number of HMM local maxima. We conclude by showing results of the method as applied to the analysis of several geophysical data sets directed at studying the seismic activity in Southern California.