Storage desk: A virtual storage system with quality of service guarantees.

摘要

The demand for storage continues its inexorable increase, driven by advances in applications and sensors that generate ever more data. In this dissertation, I present Storage desk, a virtual storage system that can aggregate a large number of distributed machines to provide storage services with quality of service guarantees. I will describe the Storage desk architecture and core components. Because storage virtualization is the prominent goal, Storage desk provides clients with the abstraction of a hard drive by utilizing the internet SCSI (iSCSI) protocol.;In Storage desk, individual users have their own individual needs for service, since they may demand different levels of quality of service (QoS) in terms of availability, reliability, capacity, performance, security, etc. Each QoS property imposes various constraints and performance trade-offs. I have utilized a market-based resource allocation model, in which pricing agents help resource providers adjust the prices as demand fluctuates. With derivative-following pricing, an agent requires no knowledge of competitors or consumers, which reduces communication overheads and avoids bottlenecks in the system. The simulation shows that, using this model, the system allows the consumers to achieve QoS goals under sufficient budgets and degrade in accordance with relative budget amounts.;As Storage desk serves clients and applications using shared storage resources, it is crucial to ensure predictable storage access even when the workloads are unknown a priori. To this end, I have taken a control-theoretic approach for automated performance control in Storage desk. Given a reference value, the feedback controller is able to regulate service requests to virtual storage resources under various scenarios. Moreover, the controller can dynamically allocate the available bandwidth among the competing clients with regards to their changing reference values.;I have developed a prototype of Storage desk that implements all the core components. I have evaluated Storage desk in two ways, benchmarks on the prototype and simulations with real-world trace data. Specifically, I used the benchmarks to measure read and write bandwidth, throughput, encryption overhead, and controller performance, and the simulations to demonstrate the feasibility of the system and the effectiveness of the market model.