Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control
The development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning p...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/16/13/4844 |
| _version_ | 1827734990892302336 |
|---|---|
| author | Wenya Xu Yanxue Li Guanjie He Yang Xu Weijun Gao |
| author_facet | Wenya Xu Yanxue Li Guanjie He Yang Xu Weijun Gao |
| author_sort | Wenya Xu |
| collection | DOAJ |
| description | The development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning process generally presents low learning efficiency using real-world stochastic data. To address this challenge, this study proposes a model-based RL approach to optimize the operation of existing zero-energy houses considering PV generation consumption and energy costs. The model-based approach takes advantage of the inner understanding of the system dynamics; this knowledge improves the learning efficiency. A reward function is designed considering the physical constraints of battery storage, photovoltaic (PV) production feed-in profit, and energy cost. Measured data of a zero-energy house are used to train and test the proposed RL agent control, including <i>Q</i>-learning, deep <i>Q</i> network (DQN), and deep deterministic policy gradient (DDPG) agents. The results show that the proposed RL agents can achieve fast convergence during the training process. In comparison with the rule-based strategy, test cases verify the cost-effectiveness performances of proposed RL approaches in scheduling operations of the hybrid energy system under different scenarios. The comparative analysis of test periods shows that the DQN agent presents better energy cost-saving performances than <i>Q</i>-learning while the <i>Q</i>-learning agent presents more flexible action control of the battery with the fluctuation of real-time electricity prices. The DDPG algorithm can achieve the highest PV self-consumption ratio, 49.4%, and the self-sufficiency ratio reaches 36.7%. The DDPG algorithm outperforms rule-based operation by 7.2% for energy cost during test periods. |
| first_indexed | 2024-03-11T01:42:58Z |
| format | Article |
| id | doaj.art-67fe6f40b4824213842c426a08f14d71 |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-11T01:42:58Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-67fe6f40b4824213842c426a08f14d712023-11-18T16:26:41ZengMDPI AGEnergies1996-10732023-06-011613484410.3390/en16134844Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning ControlWenya Xu0Yanxue Li1Guanjie He2Yang Xu3Weijun Gao4Innovation Institute for Sustainable Maritime Architecture Research and Technology, Qingdao University of Technology, Qingdao 266033, ChinaInnovation Institute for Sustainable Maritime Architecture Research and Technology, Qingdao University of Technology, Qingdao 266033, ChinaInnovation Institute for Sustainable Maritime Architecture Research and Technology, Qingdao University of Technology, Qingdao 266033, ChinaInnovation Institute for Sustainable Maritime Architecture Research and Technology, Qingdao University of Technology, Qingdao 266033, ChinaInnovation Institute for Sustainable Maritime Architecture Research and Technology, Qingdao University of Technology, Qingdao 266033, ChinaThe development of distributed renewable energy resources and smart energy management are efficient approaches to decarbonizing building energy systems. Reinforcement learning (RL) is a data-driven control algorithm that trains a large amount of data to learn control policy. However, this learning process generally presents low learning efficiency using real-world stochastic data. To address this challenge, this study proposes a model-based RL approach to optimize the operation of existing zero-energy houses considering PV generation consumption and energy costs. The model-based approach takes advantage of the inner understanding of the system dynamics; this knowledge improves the learning efficiency. A reward function is designed considering the physical constraints of battery storage, photovoltaic (PV) production feed-in profit, and energy cost. Measured data of a zero-energy house are used to train and test the proposed RL agent control, including <i>Q</i>-learning, deep <i>Q</i> network (DQN), and deep deterministic policy gradient (DDPG) agents. The results show that the proposed RL agents can achieve fast convergence during the training process. In comparison with the rule-based strategy, test cases verify the cost-effectiveness performances of proposed RL approaches in scheduling operations of the hybrid energy system under different scenarios. The comparative analysis of test periods shows that the DQN agent presents better energy cost-saving performances than <i>Q</i>-learning while the <i>Q</i>-learning agent presents more flexible action control of the battery with the fluctuation of real-time electricity prices. The DDPG algorithm can achieve the highest PV self-consumption ratio, 49.4%, and the self-sufficiency ratio reaches 36.7%. The DDPG algorithm outperforms rule-based operation by 7.2% for energy cost during test periods.https://www.mdpi.com/1996-1073/16/13/4844reinforcement learningreward designbattery storagePV consumptionenergy cost |
| spellingShingle | Wenya Xu Yanxue Li Guanjie He Yang Xu Weijun Gao Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control Energies reinforcement learning reward design battery storage PV consumption energy cost |
| title | Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control |
| title_full | Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control |
| title_fullStr | Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control |
| title_full_unstemmed | Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control |
| title_short | Performance Assessment and Comparative Analysis of Photovoltaic-Battery System Scheduling in an Existing Zero-Energy House Based on Reinforcement Learning Control |
| title_sort | performance assessment and comparative analysis of photovoltaic battery system scheduling in an existing zero energy house based on reinforcement learning control |
| topic | reinforcement learning reward design battery storage PV consumption energy cost |
| url | https://www.mdpi.com/1996-1073/16/13/4844 |
| work_keys_str_mv | AT wenyaxu performanceassessmentandcomparativeanalysisofphotovoltaicbatterysystemschedulinginanexistingzeroenergyhousebasedonreinforcementlearningcontrol AT yanxueli performanceassessmentandcomparativeanalysisofphotovoltaicbatterysystemschedulinginanexistingzeroenergyhousebasedonreinforcementlearningcontrol AT guanjiehe performanceassessmentandcomparativeanalysisofphotovoltaicbatterysystemschedulinginanexistingzeroenergyhousebasedonreinforcementlearningcontrol AT yangxu performanceassessmentandcomparativeanalysisofphotovoltaicbatterysystemschedulinginanexistingzeroenergyhousebasedonreinforcementlearningcontrol AT weijungao performanceassessmentandcomparativeanalysisofphotovoltaicbatterysystemschedulinginanexistingzeroenergyhousebasedonreinforcementlearningcontrol |