Deep Reinforcement Learning for Autonomous Water Heater Control
Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Buildings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-5309/11/11/548 |
| _version_ | 1797511041315241984 |
|---|---|
| author | Kadir Amasyali Jeffrey Munk Kuldeep Kurte Teja Kuruganti Helia Zandi |
| author_facet | Kadir Amasyali Jeffrey Munk Kuldeep Kurte Teja Kuruganti Helia Zandi |
| author_sort | Kadir Amasyali |
| collection | DOAJ |
| description | Electric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control. |
| first_indexed | 2024-03-10T05:39:48Z |
| format | Article |
| id | doaj.art-eaa8972ca3004a5e8db38130c5d468d7 |
| institution | Directory Open Access Journal |
| issn | 2075-5309 |
| language | English |
| last_indexed | 2024-03-10T05:39:48Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Buildings |
| spelling | doaj.art-eaa8972ca3004a5e8db38130c5d468d72023-11-22T22:40:06ZengMDPI AGBuildings2075-53092021-11-01111154810.3390/buildings11110548Deep Reinforcement Learning for Autonomous Water Heater ControlKadir Amasyali0Jeffrey Munk1Kuldeep Kurte2Teja Kuruganti3Helia Zandi4Computational Sciences and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37831, USAElectrification and Energy Infrastructure Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37831, USAComputational Sciences and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37831, USAComputational Sciences and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37831, USAComputational Sciences and Engineering Division, Oak Ridge National Laboratory, 1 Bethel Valley Road, Oak Ridge, TN 37831, USAElectric water heaters represent 14% of the electricity consumption in residential buildings. An average household in the United States (U.S.) spends about USD 400–600 (0.45 ¢/L–0.68 ¢/L) on water heating every year. In this context, water heaters are often considered as a valuable asset for Demand Response (DR) and building energy management system (BEMS) applications. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity cost of a water heater under a time-of-use (TOU) electricity pricing policy by only using standard DR commands. In this approach, a set of RL agents, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on an unseen pair of price and hot water usage profiles. The testing results showed that the RL agents can help save electricity cost in the range of 19% to 35% compared to the baseline operation without causing any discomfort to end users. Additionally, the RL agents outperformed rule-based and model predictive control (MPC)-based controllers and achieved comparable performance to optimization-based control.https://www.mdpi.com/2075-5309/11/11/548deep Q-networksreinforcement learningheat pump water heaterdemand responsesmart gridmachine learning |
| spellingShingle | Kadir Amasyali Jeffrey Munk Kuldeep Kurte Teja Kuruganti Helia Zandi Deep Reinforcement Learning for Autonomous Water Heater Control Buildings deep Q-networks reinforcement learning heat pump water heater demand response smart grid machine learning |
| title | Deep Reinforcement Learning for Autonomous Water Heater Control |
| title_full | Deep Reinforcement Learning for Autonomous Water Heater Control |
| title_fullStr | Deep Reinforcement Learning for Autonomous Water Heater Control |
| title_full_unstemmed | Deep Reinforcement Learning for Autonomous Water Heater Control |
| title_short | Deep Reinforcement Learning for Autonomous Water Heater Control |
| title_sort | deep reinforcement learning for autonomous water heater control |
| topic | deep Q-networks reinforcement learning heat pump water heater demand response smart grid machine learning |
| url | https://www.mdpi.com/2075-5309/11/11/548 |
| work_keys_str_mv | AT kadiramasyali deepreinforcementlearningforautonomouswaterheatercontrol AT jeffreymunk deepreinforcementlearningforautonomouswaterheatercontrol AT kuldeepkurte deepreinforcementlearningforautonomouswaterheatercontrol AT tejakuruganti deepreinforcementlearningforautonomouswaterheatercontrol AT heliazandi deepreinforcementlearningforautonomouswaterheatercontrol |