End-to-End Deep Reinforcement Learning Control for HVAC Systems in Office Buildings

摘要

The heating, ventilation, and air conditioning (HVAC) system is a major energy consumer inoffice buildings, and its operation is critical for indoor thermal comfort. While previous studies haveindicated that reinforcement learning control can improve HVAC energy efficiency, they did not provide enough information about end-to-end control (i.e., from raw observations to ready-to-implementcontrol signals) for centralized HVAC systems in multizone buildings due to the limitations of reinforcement learning methods or the test buildings being single zones with independent HVACsystems. This study developed a model-free end-to-end dynamic HVAC control method based on arecently proposed deep reinforcement learning framework to control the centralized HVAC systemof a multizone office building. By using the deep neural network, the proposed control methodcould directly take measurable parameters, including weather and indoor environment conditions,as inputs and control indoor temperature setpoints at a supervisory level. In some test cases, theproposed control method could successfully learn a dynamic control policy to reduce HVAC energyconsumption by 12.8 compared with the baseline case using conventional control methods, withoutcompromising thermal comfort. However, an over-fitting problem was noted, indicating that futurework should first focus on the generalization of deep reinforcement learning.