Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning
With the roll-out of smart meters and the increasing prevalence of distributed energy resources (DERs) at the residential level, end-users rely on home energy management systems (HEMSs) that can harness real-time data and employ artificial intelligence techniques to optimally manage the operation of...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-01-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/14/3/531 |
| _version_ | 1797409124418322432 |
|---|---|
| author | Yujian Ye Dawei Qiu Huiyu Wang Yi Tang Goran Strbac |
| author_facet | Yujian Ye Dawei Qiu Huiyu Wang Yi Tang Goran Strbac |
| author_sort | Yujian Ye |
| collection | DOAJ |
| description | With the roll-out of smart meters and the increasing prevalence of distributed energy resources (DERs) at the residential level, end-users rely on home energy management systems (HEMSs) that can harness real-time data and employ artificial intelligence techniques to optimally manage the operation of different DERs, which are targeted toward minimizing the end-user’s energy bill. In this respect, the performance of the conventional model-based demand response (DR) management approach may deteriorate due to the inaccuracy of the employed DER operating models and the probabilistic modeling of uncertain parameters. To overcome the above drawbacks, this paper develops a novel real-time DR management strategy for a residential household based on the twin delayed deep deterministic policy gradient (TD3) learning approach. This approach is model-free, and thus does not rely on knowledge of the distribution of uncertainties or the operating models and parameters of the DERs. It also enables learning of neural-network-based and fine-grained DR management policies in a multi-dimensional action space by exploiting high-dimensional sensory data that encapsulate the uncertainties associated with the renewable generation, appliances’ operating states, utility prices, and outdoor temperature. The proposed method is applied to the energy management problem for a household with a portfolio of the most prominent types of DERs. Case studies involving a real-world scenario are used to validate the superior performance of the proposed method in reducing the household’s energy costs while coping with the multi-source uncertainties through comprehensive comparisons with the state-of-the-art deep reinforcement learning (DRL) methods. |
| first_indexed | 2024-03-09T04:09:47Z |
| format | Article |
| id | doaj.art-8965c9211fbb4f7cb3987ef78231363d |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-09T04:09:47Z |
| publishDate | 2021-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-8965c9211fbb4f7cb3987ef78231363d2023-12-03T14:02:02ZengMDPI AGEnergies1996-10732021-01-0114353110.3390/en14030531Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient LearningYujian Ye0Dawei Qiu1Huiyu Wang2Yi Tang3Goran Strbac4School of Electrical Engineering, Southeast University, Nanjing 210096, ChinaDepartment of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UKSchool of Electrical Engineering, Southeast University, Nanjing 210096, ChinaSchool of Electrical Engineering, Southeast University, Nanjing 210096, ChinaDepartment of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UKWith the roll-out of smart meters and the increasing prevalence of distributed energy resources (DERs) at the residential level, end-users rely on home energy management systems (HEMSs) that can harness real-time data and employ artificial intelligence techniques to optimally manage the operation of different DERs, which are targeted toward minimizing the end-user’s energy bill. In this respect, the performance of the conventional model-based demand response (DR) management approach may deteriorate due to the inaccuracy of the employed DER operating models and the probabilistic modeling of uncertain parameters. To overcome the above drawbacks, this paper develops a novel real-time DR management strategy for a residential household based on the twin delayed deep deterministic policy gradient (TD3) learning approach. This approach is model-free, and thus does not rely on knowledge of the distribution of uncertainties or the operating models and parameters of the DERs. It also enables learning of neural-network-based and fine-grained DR management policies in a multi-dimensional action space by exploiting high-dimensional sensory data that encapsulate the uncertainties associated with the renewable generation, appliances’ operating states, utility prices, and outdoor temperature. The proposed method is applied to the energy management problem for a household with a portfolio of the most prominent types of DERs. Case studies involving a real-world scenario are used to validate the superior performance of the proposed method in reducing the household’s energy costs while coping with the multi-source uncertainties through comprehensive comparisons with the state-of-the-art deep reinforcement learning (DRL) methods.https://www.mdpi.com/1996-1073/14/3/531demand responsedistributed energy resourcesdeep neural networkdeep reinforcement learningrenewable energysmart grid |
| spellingShingle | Yujian Ye Dawei Qiu Huiyu Wang Yi Tang Goran Strbac Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning Energies demand response distributed energy resources deep neural network deep reinforcement learning renewable energy smart grid |
| title | Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning |
| title_full | Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning |
| title_fullStr | Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning |
| title_full_unstemmed | Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning |
| title_short | Real-Time Autonomous Residential Demand Response Management Based on Twin Delayed Deep Deterministic Policy Gradient Learning |
| title_sort | real time autonomous residential demand response management based on twin delayed deep deterministic policy gradient learning |
| topic | demand response distributed energy resources deep neural network deep reinforcement learning renewable energy smart grid |
| url | https://www.mdpi.com/1996-1073/14/3/531 |
| work_keys_str_mv | AT yujianye realtimeautonomousresidentialdemandresponsemanagementbasedontwindelayeddeepdeterministicpolicygradientlearning AT daweiqiu realtimeautonomousresidentialdemandresponsemanagementbasedontwindelayeddeepdeterministicpolicygradientlearning AT huiyuwang realtimeautonomousresidentialdemandresponsemanagementbasedontwindelayeddeepdeterministicpolicygradientlearning AT yitang realtimeautonomousresidentialdemandresponsemanagementbasedontwindelayeddeepdeterministicpolicygradientlearning AT goranstrbac realtimeautonomousresidentialdemandresponsemanagementbasedontwindelayeddeepdeterministicpolicygradientlearning |