An efficient compression scheme for data communication which uses a new family of self-organizing binary search trees

摘要

In this paper, we demonstrate that we can effectively use the results from the field of adaptive self-organizing data structures in enhancing compression schemes. Unlike adaptive lists, which have already been used in compression, to the best of our knowledge, adaptive self-organizing trees have not been used in this regard. To achieve this, we introduce a new data structure, the partitioning binary search tree (PBST) which, although based on the well-known binary search tree (BST), also appropriately partitions the data elements into mutually exclusive sets. When used in conjunction with Fano encoding, the PBST leads to the so-called Fano binary search tree (FBST), which, indeed, incorporates the required Fano coding (nearly equal probability) property into the BST. We demonstrate how both the PBST and the FBST can be maintained adaptively and in a self-organizing manner. The updating procedure that converts a PBST into an FBST, and the corresponding new free-based operators, namely the shift-to-left and the shift-to-right operators, are explicitly presented. The encoding and decoding procedures that also update the FBST have been implemented and rigorously tested. Our empirical results on the files of the well-known benchmarks, the Calgary and Canterbury Corpora, show that the adaptive Fano coding using FBSTs, the Huffman, and the greedy adaptive Fano coding achieve similar compression ratios. However, in terms of encoding/decoding speed, the new scheme is much faster than the latter two in the encoding phase, and they achieve approximately the same speed in the decoding phase. We believe that the same philosophy, namely that of using an adaptive self-organizing BST to maintain the frequencies, can also be utilized for other data encoding mechanisms, even as the Fenwick scheme has been used in arithmetic coding.