Efficient training of high-order hidden Markov models using first-order representations

摘要

We detail an algorithm (ORED) that transforms any higher-order hidden Markov model (HMM) to an equivalent first-order HMM. This makes it possible to process higher-order HMMs with standard techniques applicable to first-order models. Based on this equivalence, a fast incremental algorithm (FIT) is developed for training higher-order HMMs from lower-order models, thereby avoiding the training of redundant parameters. We also show that the FIT algorithm results in much faster training and better generalization compared to conventional high-order HMM approaches. This makes training of high-order HMMs practical for many applications.