Evaluation of the Efficiency of the Chi-Square Metric

摘要

The efficiency of using the chi-square metrics to weigh terms used in text documents is evaluated. The procedure includes the selection and advanced processing of class Cand ～Ctexts, compilation ofa reference dictionary and calculation of scores for all the terms in the dictionary, calculation of χ~2 coefficients for terms from a class C text, and calculation of the general efficiency factor by the sum of the coefficients found for the terms from the reference dictionary. The weighting by the χ~2 formula, odds-ratio (OR) formula, and on the basis of probabilistic variables is analyzed and compared. It was found that the best result is yielded by the OR-based weighting.