Skip to Main content Skip to Navigation
Conference papers

Integration and Evaluation of Deep Reinforcement Learning Controller in a Building Co-Simulation Environment

Abstract : Deep Reinforcement Learning (DRL) is a promising Artificial Intelligence (AI) approach for buildings heating control. However, DRL controllers require dynamic simulation involving heterogeneous physical models (building, power-grid, etc.). Co-simulation allows the inter-operation between heterogeneous components when exported following the Functional Mock-Up Interface (FMI) standard. Controllers based on DRL can be implemented using various languages but are mostly based on Python libraries like Tensorflow. Their integration into co-simulation environments requires their exportation as Functional Mockup Units (FMUs). Thus, language-specific FMI-compliant export tools are needed for every programming language, specific library or platform used. This process is costly in effort and time and results in large FMUs. This paper proposes a novel method that simplifies AI-based controller integration regardless of the language or platform in a co-simulation environment and apply our methodology to assess a DRL controller. For the first objective, we use existing FMI export tools to create an FMU having the same input and output parameters as the controller. In the proposed architecture, the FMU acts as a proxy whose objective is to communicate with an external DRL controller deployed on a local or remote machine. We propose an application of this generic architecture for heating control in a house using DACCOSIM NG co-simulation environment. Through this architecture, the deployed DRL-based controller is connected to a house energy model, which includes weather conditions and heating. We show that our proposed controller is capable to learn the system dynamics and keep temperature within 1 degree of setpoint 93% of the time.
Complete list of metadata

Cited literature [27 references]  Display  Hide  Download
Contributor : ahmed amrani Connect in order to contact the contributor
Submitted on : Sunday, October 4, 2020 - 11:39:11 AM
Last modification on : Tuesday, December 22, 2020 - 9:37:38 PM
Long-term archiving on: : Tuesday, January 5, 2021 - 6:10:57 PM


BS02020 Integ Evaluate DRL Cam...
Files produced by the author(s)


  • HAL Id : hal-02957010, version 1



Ahmed Amrani, Rim Kaddah, Jean-Philippe Tavella, Mathieu Schumann. Integration and Evaluation of Deep Reinforcement Learning Controller in a Building Co-Simulation Environment. Building Simulation and Optimization (BSO-V 2020), Sep 2020, Loughborough (virtual conference), United Kingdom. ⟨hal-02957010⟩



Record views


Files downloads