MERKLE TREE FOREST FOR SYNCHRONIZING DATA BUCKETS OF UNLIMITED SIZE IN OBJECT STORAGE SYSTEMS

摘要

Embodiments extend using sparse Merkle trees for smart synchronization of S3 buckets by overcoming fixed size limitations through creating another Merkle tree when the fixed size limit of the first tree is exceeded, and creating yet another tree when the second tree is filled up, and so on as needed. The method maintains a list of trees, in which each tree can be synchronized separately by keeping a strict division to trees according to generation number. The generation is passed from a source site to a target site during replication operations. The tagging of the generation number also makes it easy and efficient to remove an older version of an element or deal with deleted elements. This allows efficient syncing between two data object buckets without a size limitation on number of elements in a bucket.