A New Hybrid Language Independent Extraction System

摘要

Kewords extraction is identification of thematic words from a document which can depict the overall theme of the document. This paper concentrates on a new hybrid language independent keywords extraction system. Proposed hybrid keywords extraction system is hybrid of approach suggested by Bun and Ishizuka (Topic extraction from news archive using TF-PDF algorithm. In: Proceedings of third international conference on web information system engineering (WISE 02), pp 73-82) and Lee and Kim (News keyword extraction for topic tracking. In: Proceedings of fourth international conference on networked computing and advanced information management, pp 554-559 [3]). For identification of key terms from text we have used the NTF1-PSF and NTF2-PSF measures which are modified improved form of conventional TF-ISF measure. The first variant of TF is Normalized Term Frequency 1 (NTF1) which is normalized by the maximum TF in a given sentence. The NTF2 is calculated by summing up the results of dividing the frequency of a given word appears in each sentence by the frequency of all words appear in each-sentence. Proportional-sentence-frequency (PSF) of a word in a given document is the exponential of the frequency of sentences containing the word j to the total sentence-frequency in the text document. Final keywords are obtained by taking intersection of keywords sets of NTF1-PSF and NTF2-PSF and union of title keywords set obtained by title keywords extraction. The efficiency of this language independent hybrid keywords extraction system is 84.39 %.