Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm
A large number of distributed generators (GDs) such as photovoltaic panels (PVs) and energy storage (ES) systems are connected to distribution networks (DNs), and these high permeability GDs can cause voltage over-limit problems. Utilizing new developments in deep reinforcement learning, this paper...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-01-01
|
| Series: | Frontiers in Energy Research |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fenrg.2022.1097319/full |
| _version_ | 1797956164499013632 |
|---|---|
| author | Ming Ma Ming Ma Wanlin Du Ling Wang Cangbi Ding Siqi Liu |
| author_facet | Ming Ma Ming Ma Wanlin Du Ling Wang Cangbi Ding Siqi Liu |
| author_sort | Ming Ma |
| collection | DOAJ |
| description | A large number of distributed generators (GDs) such as photovoltaic panels (PVs) and energy storage (ES) systems are connected to distribution networks (DNs), and these high permeability GDs can cause voltage over-limit problems. Utilizing new developments in deep reinforcement learning, this paper proposes a multi-timescale control method for maintaining optimal voltage of a DN based on a DQN-DDPG algorithm. Here, we first analyzed the output characteristics of the devices with voltage regulation function in the DN and then used the deep Q network (DQN) algorithm to optimize the voltage regulation over longer times and the deep deterministic policy gradient (DDPG) algorithm to optimize the voltage regulation mode over short time periods. Second, the design strategy of the DQN-DDPG algorithm as based on the Markov decision process transformation was presented for the stated objectives and constraints considering the state of ES charge for prolonging the energy storage capacity. Lastly, the proposed strategy was verified on a simulation platform, and the results obtained were compared to those from a particle swarm optimization algorithm, demonstrating the method’s effectiveness. |
| first_indexed | 2024-04-10T23:44:37Z |
| format | Article |
| id | doaj.art-52b37302fd4b4453a7d884d8e201914a |
| institution | Directory Open Access Journal |
| issn | 2296-598X |
| language | English |
| last_indexed | 2024-04-10T23:44:37Z |
| publishDate | 2023-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Energy Research |
| spelling | doaj.art-52b37302fd4b4453a7d884d8e201914a2023-01-11T04:58:44ZengFrontiers Media S.A.Frontiers in Energy Research2296-598X2023-01-011010.3389/fenrg.2022.10973191097319Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithmMing Ma0Ming Ma1Wanlin Du2Ling Wang3Cangbi Ding4Siqi Liu5School of Electrical Engineering, Xi’an Jiaotong University, Xi’an, ChinaKey Laboratory of Power Quality of Guangdong Power Grid Co., Ltd., Electric Power Research Institute of Guangdong Power Grid Co., Ltd., Guangzhou, ChinaKey Laboratory of Power Quality of Guangdong Power Grid Co., Ltd., Electric Power Research Institute of Guangdong Power Grid Co., Ltd., Guangzhou, ChinaKey Laboratory of Power Quality of Guangdong Power Grid Co., Ltd., Electric Power Research Institute of Guangdong Power Grid Co., Ltd., Guangzhou, ChinaCollege of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Automation and College of Artificial Intelligence, Nanjing University of Post and Telecommunications, Nanjing, ChinaA large number of distributed generators (GDs) such as photovoltaic panels (PVs) and energy storage (ES) systems are connected to distribution networks (DNs), and these high permeability GDs can cause voltage over-limit problems. Utilizing new developments in deep reinforcement learning, this paper proposes a multi-timescale control method for maintaining optimal voltage of a DN based on a DQN-DDPG algorithm. Here, we first analyzed the output characteristics of the devices with voltage regulation function in the DN and then used the deep Q network (DQN) algorithm to optimize the voltage regulation over longer times and the deep deterministic policy gradient (DDPG) algorithm to optimize the voltage regulation mode over short time periods. Second, the design strategy of the DQN-DDPG algorithm as based on the Markov decision process transformation was presented for the stated objectives and constraints considering the state of ES charge for prolonging the energy storage capacity. Lastly, the proposed strategy was verified on a simulation platform, and the results obtained were compared to those from a particle swarm optimization algorithm, demonstrating the method’s effectiveness.https://www.frontiersin.org/articles/10.3389/fenrg.2022.1097319/fulldistributed photovoltaicdeep reinforcement learningvoltage controlmulti-timescaledistribution network |
| spellingShingle | Ming Ma Ming Ma Wanlin Du Ling Wang Cangbi Ding Siqi Liu Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm Frontiers in Energy Research distributed photovoltaic deep reinforcement learning voltage control multi-timescale distribution network |
| title | Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm |
| title_full | Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm |
| title_fullStr | Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm |
| title_full_unstemmed | Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm |
| title_short | Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm |
| title_sort | research on the multi timescale optimal voltage control method for distribution network based on a dqn ddpg algorithm |
| topic | distributed photovoltaic deep reinforcement learning voltage control multi-timescale distribution network |
| url | https://www.frontiersin.org/articles/10.3389/fenrg.2022.1097319/full |
| work_keys_str_mv | AT mingma researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm AT mingma researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm AT wanlindu researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm AT lingwang researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm AT cangbiding researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm AT siqiliu researchonthemultitimescaleoptimalvoltagecontrolmethodfordistributionnetworkbasedonadqnddpgalgorithm |