A reconfigurable stream compression hardware based on static symbol-lookup table

摘要

When we consider any applications that use large data continuously produced, it is necessary for the system developer to apply some fast method that migrates the data stream to the processors. Even if we consider the internal communications of a BigData processing system, applications that treat dataflow such as from a sensor system with tens of channels to a peripheral bus for interconnections among processing modules are currently facing a critical frequency problem to exchange data in the busses because the data size has become very large. One of the best solutions to improve the situation is to compress the exchanged data stream during the transfer in the interconnection among processing modules. However, the conventional compression mechanisms used by software solutions such as ZIP and LZW need to aggregate the compressed data and a table that includes the information for recovering the compressed data to the original one. This paper shows a novel compression mechanism based on the symbol pair matching that uses a coherent and static lookup table with a limited number of entries of the symbol pairs. Building a compression pipeline with multiple tables we can implement an effective data path of the stream-based compression with a reconfigurable and flexible compression ratio applying trained tables from the original data characteristics. This paper shows the algorithm design and an implementation example on an FPGA using the content addressable memory and reports the performance of the hardware.