Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability
The rapid development of electric vehicle (EV) technology and the consequent charging demand have brought challenges to the stable operation of distribution networks (DNs). The problem of the collaborative optimization of the charging scheduling of EVs and voltage control of the DN is intractable be...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/3/1618 |
| _version_ | 1797623160857690112 |
|---|---|
| author | Ding Liu Peng Zeng Shijie Cui Chunhe Song |
| author_facet | Ding Liu Peng Zeng Shijie Cui Chunhe Song |
| author_sort | Ding Liu |
| collection | DOAJ |
| description | The rapid development of electric vehicle (EV) technology and the consequent charging demand have brought challenges to the stable operation of distribution networks (DNs). The problem of the collaborative optimization of the charging scheduling of EVs and voltage control of the DN is intractable because the uncertainties of both EVs and the DN need to be considered. In this paper, we propose a deep reinforcement learning (DRL) approach to coordinate EV charging scheduling and distribution network voltage control. The DRL-based strategy contains two layers, the upper layer aims to reduce the operating costs of power generation of distributed generators and power consumption of EVs, and the lower layer controls the Volt/Var devices to maintain the voltage stability of the distribution network. We model the coordinate EV charging scheduling and voltage control problem in the distribution network as a Markov decision process (MDP). The model considers uncertainties of charging process caused by the charging behavior of EV users, as well as the uncertainty of uncontrollable load, system dynamic electricity price and renewable energy generation. Since the model has a dynamic state space and mixed action outputs, a framework of deep deterministic policy gradient (DDPG) is adopted to train the two-layer agent and the policy network is designed to output discrete and continuous control actions. Simulation and numerical results on the IEEE-33 bus test system demonstrate the effectiveness of the proposed method in collaborative EV charging scheduling and distribution network voltage stabilization. |
| first_indexed | 2024-03-11T09:24:43Z |
| format | Article |
| id | doaj.art-034280ede4f84fa185fabd111ef70c28 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T09:24:43Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-034280ede4f84fa185fabd111ef70c282023-11-16T18:03:41ZengMDPI AGSensors1424-82202023-02-01233161810.3390/s23031618Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage StabilityDing Liu0Peng Zeng1Shijie Cui2Chunhe Song3Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, ChinaKey Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, ChinaKey Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, ChinaKey Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, ChinaThe rapid development of electric vehicle (EV) technology and the consequent charging demand have brought challenges to the stable operation of distribution networks (DNs). The problem of the collaborative optimization of the charging scheduling of EVs and voltage control of the DN is intractable because the uncertainties of both EVs and the DN need to be considered. In this paper, we propose a deep reinforcement learning (DRL) approach to coordinate EV charging scheduling and distribution network voltage control. The DRL-based strategy contains two layers, the upper layer aims to reduce the operating costs of power generation of distributed generators and power consumption of EVs, and the lower layer controls the Volt/Var devices to maintain the voltage stability of the distribution network. We model the coordinate EV charging scheduling and voltage control problem in the distribution network as a Markov decision process (MDP). The model considers uncertainties of charging process caused by the charging behavior of EV users, as well as the uncertainty of uncontrollable load, system dynamic electricity price and renewable energy generation. Since the model has a dynamic state space and mixed action outputs, a framework of deep deterministic policy gradient (DDPG) is adopted to train the two-layer agent and the policy network is designed to output discrete and continuous control actions. Simulation and numerical results on the IEEE-33 bus test system demonstrate the effectiveness of the proposed method in collaborative EV charging scheduling and distribution network voltage stabilization.https://www.mdpi.com/1424-8220/23/3/1618electric vehicledistribution networkdeep reinforcement learningvoltage control |
| spellingShingle | Ding Liu Peng Zeng Shijie Cui Chunhe Song Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability Sensors electric vehicle distribution network deep reinforcement learning voltage control |
| title | Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability |
| title_full | Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability |
| title_fullStr | Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability |
| title_full_unstemmed | Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability |
| title_short | Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability |
| title_sort | deep reinforcement learning for charging scheduling of electric vehicles considering distribution network voltage stability |
| topic | electric vehicle distribution network deep reinforcement learning voltage control |
| url | https://www.mdpi.com/1424-8220/23/3/1618 |
| work_keys_str_mv | AT dingliu deepreinforcementlearningforchargingschedulingofelectricvehiclesconsideringdistributionnetworkvoltagestability AT pengzeng deepreinforcementlearningforchargingschedulingofelectricvehiclesconsideringdistributionnetworkvoltagestability AT shijiecui deepreinforcementlearningforchargingschedulingofelectricvehiclesconsideringdistributionnetworkvoltagestability AT chunhesong deepreinforcementlearningforchargingschedulingofelectricvehiclesconsideringdistributionnetworkvoltagestability |