Domain-specific Chinese Term Extraction via Word Segmentation Optimization

摘要

Automatic term extraction has been a research hotspot in the field of Chinese information processing. However, the existing Chinese word segmentation systems such as ICTCLAS or LTP always get a fine-grained word segmentation result which destroys the integrity of terms in the processing of domain-specific texts, so that we cannot easily get the right terms. In this work, we try to solve the problem from the angle of the optimization of Chinese word segmentation. Two algorithms of word segmentation optimization based on Frequency and word segmentation optimization based on Pointwise Mutual Information are proposed. A new term extraction method "CI-Value" to extract domain specific terms from web pages based on the proposed two word segmentation optimization algorithms is presented to solve the problem. Several groups of experiments in "SPORT", "FOOD" and "IT" domains are conducted to verify the approach's effects. The results of the experiments show that our approach has more efficiency in Chinese term extraction and solves the optimization problem of word segmentation for Chinese term extraction in open-domain such as web pages from Internet well.