A method for storing bibliometric information on items from a finite source of text, and in particular document postings for use in a full-text document retrieval system

摘要

A method to compress, store, and retrieve bibliometric information on multiple sources of text is presented. The compression consists of 2 parts, and may use any one of the many ordering-based bibliometric laws for sources of text. The first compression part comprises of the storage of bibliometric information on the items from a text source, using the rank of the items in the ordering relation as defined in the bibliometric law as an indication of the bibliometric information. The second compression part efficiently uses pointers and tables to get rid of redundant information. As an application, a posting compression method is presented for use in term weighting retrieval systems. The first compression uses a postulated rank-occurrence frequency relation for the document in question that has as only variable the document's length, for example Zipf's law that states that the product of rank and frequency is approximately constant. The second compression part efficiently uses pointers and a few tables next to the principal storage. The compression makes use of direct random addressability. All postings relating to a particular document may be stored together, allowing easy expendability and updating. With respect to conventional technology, storage requirements is roughly halved.