The Semantic Level of Shannon Information: Are Highly Informative Words Good Keywords? A Study on German

摘要

This paper reports the results of a study on automatic keyword extraction in German. We employed in general two types of methods: (A) unsupervised, based on information theory, i.e., (ⅰ) a bigram model, (ⅱ) a probabilistic parser model, and (ⅲ) a novel model which considers topics within the discourse of target word for the calculation of their information content, and (B) supervised, employing a recurrent neural network (RNN). As baselines, we employed TextRank and the TF-IDF ranking function. The topic model (A)(ⅲ) outperformed clearly all remaining models, even TextRank and TF-IDF. In contrast, RNN performed poorly. We take the results as first evidence that (ⅰ) information content can be employed for keyword extraction tasks and has thus a clear correspondence to semantics of natural language, and (ⅱ) that-as a cognitive principle-the information content of words is determined from extra-sentential contexts, i.e., from the discourse of words.