Improving the Compression of Inverted Index using Sequitur Algorithm

摘要

Encoding methods provide a large scale compression to huge inverted files index. This type of index is mostly used in search engines because it allows fast query resolution and good memory utilization. There is always trade off between cost of storing the compressed index and cost of using it during query evaluating. This paper proposes a technique to enhance the compression ratio. This new technique is based on sequitur algorithm that is used in mainly in grammar compression.