A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters
The deep reinforcement learning (DRL) technique has gained attention for its potential in designing “virtual network” controllers. This skill utilizes a novel solution that can avoid the specific parameters and system model required in classical dynamic programming algorithms. However, addressing th...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/10/5879 |
| _version_ | 1797601298055430144 |
|---|---|
| author | Mu Yang Xiaojie Wu Maxwell Chiemeka Loveth |
| author_facet | Mu Yang Xiaojie Wu Maxwell Chiemeka Loveth |
| author_sort | Mu Yang |
| collection | DOAJ |
| description | The deep reinforcement learning (DRL) technique has gained attention for its potential in designing “virtual network” controllers. This skill utilizes a novel solution that can avoid the specific parameters and system model required in classical dynamic programming algorithms. However, addressing the issue of system uncertainties and performance deterioration remains a challenge. To overcome this challenge, the authors propose a new control prototype using a twin delayed deep deterministic policy gradient (TD3)-based adaptive controller, which replaces the conventional virtual synchronous generator (VSG) module in the modular multilevel converter (MMC) control. In this approach, an adaptive programming module is developed using a critic fuzzy network point of view to determine the optimal control policy. The modification presented in this framework is able to improve the system stability and resist disruptions while retaining the merits of the conventional VSG control model. The proposed approach is implemented and tested using the DRL toolbox in MATLAB/Simulink. |
| first_indexed | 2024-03-11T04:00:02Z |
| format | Article |
| id | doaj.art-f219e23c79e4446482dcbaa2c347178a |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T04:00:02Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-f219e23c79e4446482dcbaa2c347178a2023-11-18T00:17:06ZengMDPI AGApplied Sciences2076-34172023-05-011310587910.3390/app13105879A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel ConvertersMu Yang0Xiaojie Wu1Maxwell Chiemeka Loveth2School of Electrical Engineering, China University of Mining and Technology, Xuzhou 221000, ChinaSchool of Electrical Engineering, China University of Mining and Technology, Xuzhou 221000, ChinaSchool of Electrical Engineering, China University of Mining and Technology, Xuzhou 221000, ChinaThe deep reinforcement learning (DRL) technique has gained attention for its potential in designing “virtual network” controllers. This skill utilizes a novel solution that can avoid the specific parameters and system model required in classical dynamic programming algorithms. However, addressing the issue of system uncertainties and performance deterioration remains a challenge. To overcome this challenge, the authors propose a new control prototype using a twin delayed deep deterministic policy gradient (TD3)-based adaptive controller, which replaces the conventional virtual synchronous generator (VSG) module in the modular multilevel converter (MMC) control. In this approach, an adaptive programming module is developed using a critic fuzzy network point of view to determine the optimal control policy. The modification presented in this framework is able to improve the system stability and resist disruptions while retaining the merits of the conventional VSG control model. The proposed approach is implemented and tested using the DRL toolbox in MATLAB/Simulink.https://www.mdpi.com/2076-3417/13/10/5879virtual synchronous machine (VSG)deep reinforcement learning (DRL)modular multilevel converter (MMC)twin delayed deep deterministic policy gradient (TD3) |
| spellingShingle | Mu Yang Xiaojie Wu Maxwell Chiemeka Loveth A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters Applied Sciences virtual synchronous machine (VSG) deep reinforcement learning (DRL) modular multilevel converter (MMC) twin delayed deep deterministic policy gradient (TD3) |
| title | A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters |
| title_full | A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters |
| title_fullStr | A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters |
| title_full_unstemmed | A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters |
| title_short | A Deep Reinforcement Learning Design for Virtual Synchronous Generators Accommodating Modular Multilevel Converters |
| title_sort | deep reinforcement learning design for virtual synchronous generators accommodating modular multilevel converters |
| topic | virtual synchronous machine (VSG) deep reinforcement learning (DRL) modular multilevel converter (MMC) twin delayed deep deterministic policy gradient (TD3) |
| url | https://www.mdpi.com/2076-3417/13/10/5879 |
| work_keys_str_mv | AT muyang adeepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters AT xiaojiewu adeepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters AT maxwellchiemekaloveth adeepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters AT muyang deepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters AT xiaojiewu deepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters AT maxwellchiemekaloveth deepreinforcementlearningdesignforvirtualsynchronousgeneratorsaccommodatingmodularmultilevelconverters |