LOG-STRUCTURED MERGE TREE BASED DATA STORAGE ARCHITECTURE

摘要

A system, method and program product for implementing an LSM tree data store in a storage infrastructure. A system is disclosed having: a system for handling read requests and write requests using a key-value pair index to store and retrieve data in an LSM data store; and a compaction manager that reorganizes data in the LSM data store using: a partition system that (a) partitions a first level file into a set of subfiles when a partition threshold is exceeded, and (b) stores the subfiles from the first level file in an intermediate level between the first level and a second level, wherein the subfiles are partitioned by range to correspond with files in the second level; and a merge system that merges a group of files comprising a second level file with one or more corresponding subfiles when a merge threshold is exceeded.