Language identification in web documents using discrete HMMs

摘要

This paper deals with language identification in the domain of web documents. The proposed system is built on hidden Markov models (HMMs) that enable the modeling of character sequences. Furthermore, the use of HMMs provides the means for language tracking, that is, language identification across the segments of a multilingual document. (C) 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.