Flexible Transmission Network Expansion Planning Based on DQN Algorithm
Compared with static transmission network expansion planning (TNEP), multi-stage TNEP is more in line with the actual situation, but the modeling is also more complicated. This paper proposes a new multi-stage TNEP method based on the deep <i>Q</i>-network (DQN) algorithm, which can solv...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/14/7/1944 |
| _version_ | 1797539210267197440 |
|---|---|
| author | Yuhong Wang Lei Chen Hong Zhou Xu Zhou Zongsheng Zheng Qi Zeng Li Jiang Liang Lu |
| author_facet | Yuhong Wang Lei Chen Hong Zhou Xu Zhou Zongsheng Zheng Qi Zeng Li Jiang Liang Lu |
| author_sort | Yuhong Wang |
| collection | DOAJ |
| description | Compared with static transmission network expansion planning (TNEP), multi-stage TNEP is more in line with the actual situation, but the modeling is also more complicated. This paper proposes a new multi-stage TNEP method based on the deep <i>Q</i>-network (DQN) algorithm, which can solve the multi-stage TNEP problem based on a static TNEP model. The main purpose of this research is to provide grid planners with a simple and effective multi-stage TNEP method, which is able to flexibly adjust the network expansion scheme without replanning. The proposed method takes into account the construction sequence of lines in the planning and completes the adaptive planning of lines by utilizing the interactive learning characteristics of the DQN algorithm. In order to speed up the learning efficiency of the algorithm and enable the agent to have a better judgment on the reward of the line-building action, the prioritized experience replay (PER) strategy is added to the DQN algorithm. In addition, the economy, reliability, and flexibility of the expansion scheme are considered in order to evaluate the scheme more comprehensively. The fault severity of equipment is considered on the basis of the Monte Carlo method to obtain a more comprehensive system state simulation. Finally, extensive studies are conducted with IEEE 24-bus reliability test system, and the computational results demonstrate the effectiveness and adaptability of the proposed flexible TNEP method. |
| first_indexed | 2024-03-10T12:42:01Z |
| format | Article |
| id | doaj.art-d6810e1d713248c2830f2c0d1bb497be |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-10T12:42:01Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-d6810e1d713248c2830f2c0d1bb497be2023-11-21T13:47:53ZengMDPI AGEnergies1996-10732021-04-01147194410.3390/en14071944Flexible Transmission Network Expansion Planning Based on DQN AlgorithmYuhong Wang0Lei Chen1Hong Zhou2Xu Zhou3Zongsheng Zheng4Qi Zeng5Li Jiang6Liang Lu7College of Electrical Engineering, Sichuan University, Chengdu 610065, ChinaCollege of Electrical Engineering, Sichuan University, Chengdu 610065, ChinaState Grid Southwest China Branch, Chengdu 610041, ChinaCollege of Electrical Engineering, Sichuan University, Chengdu 610065, ChinaCollege of Electrical Engineering, Sichuan University, Chengdu 610065, ChinaCollege of Electrical Engineering, Sichuan University, Chengdu 610065, ChinaState Grid Southwest China Branch, Chengdu 610041, ChinaState Grid Southwest China Branch, Chengdu 610041, ChinaCompared with static transmission network expansion planning (TNEP), multi-stage TNEP is more in line with the actual situation, but the modeling is also more complicated. This paper proposes a new multi-stage TNEP method based on the deep <i>Q</i>-network (DQN) algorithm, which can solve the multi-stage TNEP problem based on a static TNEP model. The main purpose of this research is to provide grid planners with a simple and effective multi-stage TNEP method, which is able to flexibly adjust the network expansion scheme without replanning. The proposed method takes into account the construction sequence of lines in the planning and completes the adaptive planning of lines by utilizing the interactive learning characteristics of the DQN algorithm. In order to speed up the learning efficiency of the algorithm and enable the agent to have a better judgment on the reward of the line-building action, the prioritized experience replay (PER) strategy is added to the DQN algorithm. In addition, the economy, reliability, and flexibility of the expansion scheme are considered in order to evaluate the scheme more comprehensively. The fault severity of equipment is considered on the basis of the Monte Carlo method to obtain a more comprehensive system state simulation. Finally, extensive studies are conducted with IEEE 24-bus reliability test system, and the computational results demonstrate the effectiveness and adaptability of the proposed flexible TNEP method.https://www.mdpi.com/1996-1073/14/7/1944flexible transmission network expansion planningdeep Q-networkprioritized experience replay strategyconstruction sequence |
| spellingShingle | Yuhong Wang Lei Chen Hong Zhou Xu Zhou Zongsheng Zheng Qi Zeng Li Jiang Liang Lu Flexible Transmission Network Expansion Planning Based on DQN Algorithm Energies flexible transmission network expansion planning deep Q-network prioritized experience replay strategy construction sequence |
| title | Flexible Transmission Network Expansion Planning Based on DQN Algorithm |
| title_full | Flexible Transmission Network Expansion Planning Based on DQN Algorithm |
| title_fullStr | Flexible Transmission Network Expansion Planning Based on DQN Algorithm |
| title_full_unstemmed | Flexible Transmission Network Expansion Planning Based on DQN Algorithm |
| title_short | Flexible Transmission Network Expansion Planning Based on DQN Algorithm |
| title_sort | flexible transmission network expansion planning based on dqn algorithm |
| topic | flexible transmission network expansion planning deep Q-network prioritized experience replay strategy construction sequence |
| url | https://www.mdpi.com/1996-1073/14/7/1944 |
| work_keys_str_mv | AT yuhongwang flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT leichen flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT hongzhou flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT xuzhou flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT zongshengzheng flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT qizeng flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT lijiang flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm AT lianglu flexibletransmissionnetworkexpansionplanningbasedondqnalgorithm |