Use of Generalized Context Trees, a means for assigning a unique state from a finite set to any string, is provided. The method optionally refines the generalized context tree into a refined generalized context tree having a finite state machine (FSM) property. Refining occurs whenever the generalized context tree does not have the finite state machine property. Alternately, a method for constructing a representation of a source usable within an FSM is provided, comprising evaluating a node comprising a suffix tail and verifying the suffix tail is included in the representation, and inserting at least one node to the representation when the suffix tail is not in the representation.
展开▼