Chinese sentence compression based on statistics probability and dependency analysis

摘要

In this paper we describe a Chinese sentence compression tool, which makes use of various technologies. We first describe how to prepare our own Chinese training corpus and from what we learn the removal and non-removal probabilities. Then we bring in a knowledge base to preprocess the input sentences. Following that, how an input sentence gets analyzed is described, using Chinese word segmentation, a pos-tagging removal program and a shallow parser which outputs the collapsed dependencies. Next, we use the removal probabilities and compression rules to finish the task. Experimental results have shown that this method is feasible.