Lone Star Stack: Architecture of a Disk-Based Archival System

摘要

The need for huge storage systems rises with the ever growing creation of data. With growing capacities and shrinking prices, "write once read sometimes" workloads become more common. New data is constantly added, rarely updated or deleted, and every stored byte might be read at any time - a common pattern for digital archives or big data scenarios. We present the Lone Star Stack, a disk based archival storage system building block that is optimized for high reliability and energy efficiency. It provides a POSIX file system interface that uses flash based storage for write-offloading and metadata and the disk-based Lone Star RAID for user data storage. The RAID attempts to spin down disks as soon and as long as possible. For reads, only a single disk is accessed, while writes require 3 additional parity disks to be spun up. The cache aggregates new files and a semantic data placement engine decides how they are persisted to the RAID. Asynchronous data movers then persist the data. The system provides an end-to-end data integrity, an elastic fault tolerance that can at least recover from all 3-disk failures, and provides multiple paths for data integrity checking and recovery. The system can use 70% of the raw disk capacity and is optimized for fast reads with a minimum number of powered on disk drives.