Dynamic power management for computing systems in partially observable environment.

摘要

Power consumption has become a big concern in today's computing system. Excessive power consumption increases the die temperature, reduces chip reliability and also degrades system performance. High power consumption also increases the cost for energy as well as the cost for cooling devices. Most of the state-of-art system modules are capable of trading power for performance or being put into sleep or low power mode to reduce power consumption. Runtime dynamic power management has become a popular technique for power reduction at system level. The power manager observes the system status and selectively shuts-off or slows-down those system components that are idle or underutilized. However, as the hardware and software complexity grows, it is unlikely for the power manager to have a full observation of the entire system status.;In this dissertation, we consider efficient dynamic power management in a partially observable environment. We first discuss three scenarios of partial observations that may occur in an embedded system and their modeling techniques are presented. We then present a complete framework of modeling and optimization for stochastic power management using Hidden Markov Model (HMM) and Partially Observable Markov Decision Process (POMDP). The proposed technique discovers the HMM of the workload by maximizing the likelihood of the observed attribute sequence. The POMDP optimization is formulated and solved as a quadraticly constrained linear programming (QCLP). We further present a novel on-line power management technique based on model-free constrained reinforcement learning (RL). It learns the best power management policy that gives the minimum power consumption for a given performance constraint without any prior information of workload. Compared with existing machine learning based power management techniques, the RL based learning is capable of exploring the trade-off in the power-performance design space and converging to a better power management policy. Experimental results show that the proposed RL based power management achieves 24% and 3% reduction in power and latency respectively comparing to the existing expert based power management.