An AGC Dynamic Optimization Method Based on Proximal Policy Optimization
The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctua...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-07-01
|
| Series: | Frontiers in Energy Research |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fenrg.2022.947532/full |
| _version_ | 1811225363609550848 |
|---|---|
| author | Zhao Liu Jiateng Li Pei Zhang Zhenhuan Ding Yanshun Zhao |
| author_facet | Zhao Liu Jiateng Li Pei Zhang Zhenhuan Ding Yanshun Zhao |
| author_sort | Zhao Liu |
| collection | DOAJ |
| description | The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this article proposes a novel framework based on proximal policy optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions and state and action space designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system and compared with the classical proportional−integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show that the framework proposed in this article can make the frequency characteristic better satisfy the control performance standard (CPS) under the scenario of large fluctuations in power systems. |
| first_indexed | 2024-04-12T09:05:49Z |
| format | Article |
| id | doaj.art-41772396ce5b452d984560c506e7a914 |
| institution | Directory Open Access Journal |
| issn | 2296-598X |
| language | English |
| last_indexed | 2024-04-12T09:05:49Z |
| publishDate | 2022-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Energy Research |
| spelling | doaj.art-41772396ce5b452d984560c506e7a9142022-12-22T03:39:06ZengFrontiers Media S.A.Frontiers in Energy Research2296-598X2022-07-011010.3389/fenrg.2022.947532947532An AGC Dynamic Optimization Method Based on Proximal Policy OptimizationZhao Liu0Jiateng Li1Pei Zhang2Zhenhuan Ding3Yanshun Zhao4School of Electrical Engineering, Beijing Jiaotong University, Beijing, ChinaDepartment of Artificial Intelligence Applications, China Electric Power Research Institute, Beijing, ChinaSchool of Electrical Engineering, Beijing Jiaotong University, Beijing, ChinaSchool of Artificial Intelligence, Anhui University, Hefei, ChinaSchool of Electrical Engineering, Beijing Jiaotong University, Beijing, ChinaThe increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for automatic generation control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this article proposes a novel framework based on proximal policy optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions and state and action space designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system and compared with the classical proportional−integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show that the framework proposed in this article can make the frequency characteristic better satisfy the control performance standard (CPS) under the scenario of large fluctuations in power systems.https://www.frontiersin.org/articles/10.3389/fenrg.2022.947532/fullautomatic generation controladvanced optimization strategydeep reinforcement learningrenewable energyproximal policy optimization |
| spellingShingle | Zhao Liu Jiateng Li Pei Zhang Zhenhuan Ding Yanshun Zhao An AGC Dynamic Optimization Method Based on Proximal Policy Optimization Frontiers in Energy Research automatic generation control advanced optimization strategy deep reinforcement learning renewable energy proximal policy optimization |
| title | An AGC Dynamic Optimization Method Based on Proximal Policy Optimization |
| title_full | An AGC Dynamic Optimization Method Based on Proximal Policy Optimization |
| title_fullStr | An AGC Dynamic Optimization Method Based on Proximal Policy Optimization |
| title_full_unstemmed | An AGC Dynamic Optimization Method Based on Proximal Policy Optimization |
| title_short | An AGC Dynamic Optimization Method Based on Proximal Policy Optimization |
| title_sort | agc dynamic optimization method based on proximal policy optimization |
| topic | automatic generation control advanced optimization strategy deep reinforcement learning renewable energy proximal policy optimization |
| url | https://www.frontiersin.org/articles/10.3389/fenrg.2022.947532/full |
| work_keys_str_mv | AT zhaoliu anagcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT jiatengli anagcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT peizhang anagcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT zhenhuanding anagcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT yanshunzhao anagcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT zhaoliu agcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT jiatengli agcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT peizhang agcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT zhenhuanding agcdynamicoptimizationmethodbasedonproximalpolicyoptimization AT yanshunzhao agcdynamicoptimizationmethodbasedonproximalpolicyoptimization |