Using Statistical Properties for Author Identification

摘要

Languages in general are highly redundant which makes text highly compressible. In this paper, English language redundancy is exploited to predict the author of English text. The method developed starts by training the system using texts with known authors. Distinct blocks for texts written by each author are determined. Those blocks are then filtered to produce, for each author, a set of unique blocks that occur in his/her writings but not in other authors' texts. In the normal operation mode, text to be categorized is processed to determine the distinct blocks in that text. A comparison between this set of distinct blocks and the unique set of distinct blocks for each author results in correct author categorization. The method described in this paper was proven to work successfully in text classification and author categorization and has the potential to be a universal method since it was tested on English and Arabic texts.