Designing distributed systems for intermittent power.

摘要

The increasing demand for computing infrastructure, such as data centers and storage systems, has increased their energy footprint. As a result of this growth, computing infrastructure today contribute 2-3% of the global carbon emissions. Furthermore, the energy-related costs have now become a significant fraction of the total cost of ownership (TCO) of a modern computing infrastructure. Hence, to reduce the financial and environmental impact of growing energy demands the design of eco-friendly green infrastructure has become an important societal need. This thesis focuses on designing distributed systems, primarily data centers and storage systems, to run on renewable energy sources such as solar and wind.;As renewables are intermittent in nature, accurate predictions of future energy is important for a distributed system to balance workload demand and energy supply, and optimize its performance amid significant and frequent changes in both demand and supply. To accurately predict energy harvesting in all weather conditions, I develop two prediction models that leverage weather forecasts to predict solar and wind energy harvesting. The first prediction model is an empirical model that uses sky cover forecast and wind speed forecast to predict solar energy and wind energy, respectively, in the future. The second prediction model is a machine learning based model that uses statistical power of machine learning techniques to give better predictions of solar energy harvesting than the empirical model.;To regulate the energy footprint of a server I propose a new energy abstraction, called Blink, that applies duty cycle to the server to cap power consumption to supply. I also propose several blinking policies to coordinate blinking across servers to regulate cluster-wide power consumption with changes in the available power. Further, I show that a real-world application can be redesigned, with modest complexity, to perform well on intermittent power.;To extend the applicability of blinking beyond an in-memory cache server I use the blinking abstraction to design two different distributed systems—(a) Distributed File System, and (b) Multimedia Cache—for intermittent power. I propose several design techniques, including a staggered blinking policy and power-balanced data layout, to optimize the performance of these systems under intermittent power scenarios. Additionally, I experiment with three unmodified real-world applications—(a) Memcache, (b) MapReduce, and (c) Search Engine—to test the practicality of our blink-aware file system. Our results show that real-world applications can perform reasonably well for real workloads in spite of significant and frequent variations in power supply. Finally, I use a real WiMAX testbed to demonstrate that our blink-aware multimedia cache can significantly save bandwidth usage of cell towers while providing good performance under intermittent power constraints.