Adaptive Compression Algorithm Selection Using LSTM Network in Column-oriented Database

摘要

Data compression is a key part of database management systems for storage saving and performance enhancement. In column-oriented databases, records belong to the same attribute are stored nearby, and the similarity between these records increases the compressibility of data and expands the range of compression algorithms to choose. Since different data compression algorithms process data in different manners, the achieved compression ratio varies significantly. This makes it worth studying the choice of compression algorithms depending on features of data to be compressed. As Recurrent Neural Networks is good at processing and making predictions based on series of data, we propose a Long-Short Term Memory network based model to select compression algorithm for input data blocks adaptively. Given a typical database benchmark, we implemented our model to formulate compression strategies for each data block and managed to reduce at most 15% storage size than using a single compression algorithm scheme.