A Comparative Evaluation of Different Keyword Extraction Techniques

摘要

Retrieving keywords in a text has been attracting researchers for a long time as it forms a base for many natural language applications like information retrieval, text summarization, document categorization, etc. A text is a collection of words that represent the theme of the text naturally and to bring the naturalism under certain rules is itself a challenging task. In the present paper, the author evaluate different spatial distribution-based keyword extraction methods available in the literature on three standard scientific texts. The author choose the first few high-frequency words for evaluation to reduce the complexity as all the methods are somehow based on frequency. The author find that the methods are not providing good results particularly in the case of the first few retrieved words. Thus, the author propose a new measure based on frequency, inverse document frequency, variance, and Tsallis entropy. Evaluation of different methods is done on the basis of precision, recall, and F-measure. Results show that the proposed method provides improved results.