Method for estimation of feature gain and training starting point for maximum entropy/minimum divergence probability models

摘要

A method and apparatus for efficiently determining the gain of a feature function in a maximum entropy/minimum divergence probability model in a single pass through a training corpus. A method for determining the gain of a feature in such a model includes the steps of a selecting a set of evaluation points and determining the value of a function referred to as the gainsum derivative at each of the evaluation points. An approximation function which can be evaluated at substantially any point in a continuous domain is then selected based upon the discrete values of the gainsum derivative at the evaluation points. The approximation function is then employed to determine the argument value that maximizes an approximated gain function. The approximate gain value is then determined by evaluating the approximated gain function at this argument value. The apparatus of the present invention includes means for performing the steps of the disclosed method.