Improving Quality of the Web Corpus

摘要

We present a method for constructing a text corpus from the Web. We use a collection of Web pages so that we can solve lack of resource amount. However, some of the Web texts have a low quality. We then propose some procedures to reduce these low-quality texts out of the Web corpus. We have proved in the experiments that our Web corpus outperformed a newspaper corpus of the same size, in terms of number of words and case frame variations. We also illustrate that our Web corpus is also superior to unprocessed Web texts.