MARCO - Multi-Agent Reinforcement learning based COntrol of building HVAC systems

Srinarayana Nagarathinam, Vishnu Menon, Arunchandar Vasan, Anand Sivasubramaniam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Optimal control of building heating, ventilation, air-conditioning (HVAC) equipment has typically been based on rules and model-based predictive control (MPC). Challenges in developing accurate models of buildings render these approaches sub-optimal and unstable in real-life operations. Model-free Deep Reinforcement Learning (DRL) approaches have been proposed very recently to address this. However, existing works on DRL for HVAC suffer from some limitations. First, they consider buildings with few HVAC units, thus leaving open the question of scale. Second, they consider only air-side control of air-handling-units (AHUs) without taking into the water-side chiller control, though chillers account for a significant portion of HVAC energy. Third, they use a single learning agent that adjusts multiple set-points of the HVAC system. We present MARCO - Multi-Agent Reinforcement learning COntrol for HVACs that addresses these challenges. Our approach achieves scale by transfer of learning across HVAC sub-systems. MARCO uses separate DRL agents that control both the AHUs and chillers to jointly optimize HVAC operations. We train and evaluate MARCO on a simulation environment with real-world configurations. We show that MARCO performs better than the as-is HVAC control strategy. We find that MARCO achieves performance comparable to an MPC Oracle that has perfect system knowledge; and better than MPC suffering from systemic calibration uncertainties. Other key findings from our evaluation studies include the following: 1) distributed agents perform significantly better than a central agent for HVAC control; 2) cooperative agents improve over competing agents; and 3) domain knowledge can be exploited to reduce the training time significantly.

Original languageEnglish (US)
Title of host publicatione-Energy 2020 - Proceedings of the 11th ACM International Conference on Future Energy Systems
PublisherAssociation for Computing Machinery, Inc
Pages57-67
Number of pages11
ISBN (Electronic)9781450380096
DOIs
StatePublished - Jun 12 2020
Event11th ACM International Conference on Future Energy Systems, e-Energy 2020 - Virtual, Australia
Duration: Jun 22 2020Jun 26 2020

Publication series

Namee-Energy 2020 - Proceedings of the 11th ACM International Conference on Future Energy Systems

Conference

Conference11th ACM International Conference on Future Energy Systems, e-Energy 2020
CountryAustralia
CityVirtual
Period6/22/206/26/20

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Energy Engineering and Power Technology

Fingerprint Dive into the research topics of 'MARCO - Multi-Agent Reinforcement learning based COntrol of building HVAC systems'. Together they form a unique fingerprint.

Cite this