Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning
In terms of model-free voltage control methods, when the device or topology of the system changes, the model's accuracy often decreases, so an adaptive model is needed to coordinate the changes of input. To overcome the defects of a model-free control method, this paper proposes an automatic vo...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/169612 |
| _version_ | 1811690499962044416 |
|---|---|
| author | Wang, Tianjing Tang, Yong |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Wang, Tianjing Tang, Yong |
| author_sort | Wang, Tianjing |
| collection | NTU |
| description | In terms of model-free voltage control methods, when the device or topology of the system changes, the model's accuracy often decreases, so an adaptive model is needed to coordinate the changes of input. To overcome the defects of a model-free control method, this paper proposes an automatic voltage control (AVC) method for differential power grids based on transfer learning and deep reinforcement learning. First, when constructing the Markov game of AVC, both the magnitude and number of voltage deviations are taken into account in the reward. Then, an AVC method based on constrained multi-agent deep reinforcement learning (DRL) is developed. To further improve learning efficiency, domain knowledge is used to reduce action space. Next, distribution adaptation transfer learning is introduced for the AVC transfer circumstance of systems with the same structure but distinct topological relations/parameters, which can perform well without any further training even if the structure changes. Moreover, for the AVC transfer circumstance of various power grids, parameter-based transfer learning is created, which enhances the target system's training speed and effect. Finally, the method's efficacy is tested using two IEEE systems and two real-world power grids. |
| first_indexed | 2024-10-01T06:04:59Z |
| format | Journal Article |
| id | ntu-10356/169612 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:04:59Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1696122023-07-28T15:40:00Z Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning Wang, Tianjing Tang, Yong School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Deep Reinforcement Learning Differential Power Grids In terms of model-free voltage control methods, when the device or topology of the system changes, the model's accuracy often decreases, so an adaptive model is needed to coordinate the changes of input. To overcome the defects of a model-free control method, this paper proposes an automatic voltage control (AVC) method for differential power grids based on transfer learning and deep reinforcement learning. First, when constructing the Markov game of AVC, both the magnitude and number of voltage deviations are taken into account in the reward. Then, an AVC method based on constrained multi-agent deep reinforcement learning (DRL) is developed. To further improve learning efficiency, domain knowledge is used to reduce action space. Next, distribution adaptation transfer learning is introduced for the AVC transfer circumstance of systems with the same structure but distinct topological relations/parameters, which can perform well without any further training even if the structure changes. Moreover, for the AVC transfer circumstance of various power grids, parameter-based transfer learning is created, which enhances the target system's training speed and effect. Finally, the method's efficacy is tested using two IEEE systems and two real-world power grids. Published version 2023-07-26T02:53:57Z 2023-07-26T02:53:57Z 2023 Journal Article Wang, T. & Tang, Y. (2023). Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning. CSEE Journal of Power and Energy Systems, 9(3), 937-948. https://dx.doi.org/10.17775/CSEEJPES.2021.06320 2096-0042 https://hdl.handle.net/10356/169612 10.17775/CSEEJPES.2021.06320 2-s2.0-85162094252 3 9 937 948 en CSEE Journal of Power and Energy Systems © 2021 CSEE. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). application/pdf |
| spellingShingle | Engineering::Electrical and electronic engineering Deep Reinforcement Learning Differential Power Grids Wang, Tianjing Tang, Yong Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title | Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title_full | Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title_fullStr | Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title_full_unstemmed | Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title_short | Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| title_sort | automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning |
| topic | Engineering::Electrical and electronic engineering Deep Reinforcement Learning Differential Power Grids |
| url | https://hdl.handle.net/10356/169612 |
| work_keys_str_mv | AT wangtianjing automaticvoltagecontrolofdifferentialpowergridsbasedontransferlearninganddeepreinforcementlearning AT tangyong automaticvoltagecontrolofdifferentialpowergridsbasedontransferlearninganddeepreinforcementlearning |