Dynamic Storage Method of Big Data Based on Layered and Configurable Technology

摘要

In big data era, how to reasonably divide various types of data and efficiently store so large and complex data sets is an important challenge. This paper proposes a layered and configurable storage model to improve the storage capability of big data. First, three-layer hybrid row-column-store storage model is presented, which contains metadata definition layer, key-value model layer, and data physical storage layer. This model combines the advantages of row-store and column-store. Second, based on the three-layer storage model, the realization process is presented. According to the characteristics of each column of data, the suitable storage mode of row-store or column-store is chosen. The proposed model can provide a new technology support for the storage of big data.