AUTOMATIC EXTRACTION OF THE UNLISTED TERMS IN THE FIELD OF INFORMATION TECHNOLOGY BASED ON THE DYNAMIC CIRCULATION CORPUS

摘要

This paper discusses automatic extraction of the unlisted terms in the field of Information Technology based on the large-scale DCC (Dynamic Circulation Corpus), under the theory of Dynamic Updating of Language and Knowledge. It proposes the concept of Concatenation Index to decide whether a character string is a word/phrase or not. It also presents a new approach named "Concatenation Index + TFIDF" for extracting unlisted terms in large scale corpus of a certain field. The experiment selects the texts, around 17 million Chinese characters, in the field of IT (Information Technology) as the object corpus; and the texts, around 600 million Chinese characters, in the field of common usage as the contrast corpus. As a result, the tentative work flow has been established, and the approach turned out to be efficient.