Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme
In the smart grid environment, the penetration of electric vehicle (EV) is increasing, and dynamic pricing and vehicle-to-grid technologies are being introduced. Consequently, automatic charging and discharging scheduling responding to electricity prices that change over time is required to reduce t...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-04-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/13/8/1950 |
| _version_ | 1797570527969148928 |
|---|---|
| author | Jaehyun Lee Eunjung Lee Jinho Kim |
| author_facet | Jaehyun Lee Eunjung Lee Jinho Kim |
| author_sort | Jaehyun Lee |
| collection | DOAJ |
| description | In the smart grid environment, the penetration of electric vehicle (EV) is increasing, and dynamic pricing and vehicle-to-grid technologies are being introduced. Consequently, automatic charging and discharging scheduling responding to electricity prices that change over time is required to reduce the charging cost of EVs, while increasing the grid reliability by moving charging loads from on-peak to off-peak periods. Hence, this study proposes a deep reinforcement learning-based, real-time EV charging and discharging algorithm. The proposed method utilizes kernel density estimation, particularly the nonparametric density function estimation method, to model the usage pattern of a specific charger at a specific location. Subsequently, the estimated density function is used to sample variables related to charger usage pattern so that the variables can be cast in the training process of a reinforcement learning agent. This ensures that the agent optimally learns the characteristics of the target charger. We analyzed the effectiveness of the proposed algorithm from two perspectives, i.e., charging cost and load shifting effect. Simulation results show that the proposed method outperforms the benchmarks that simply model usage pattern through general assumptions in terms of charging cost and load shifting effect. This means that when a reinforcement learning-based charging/discharging algorithm is deployed in a specific location, it is better to use data-driven approach to reflect the characteristics of the location, so that the charging cost reduction and the effect of load flattening are obtained. |
| first_indexed | 2024-03-10T20:26:47Z |
| format | Article |
| id | doaj.art-130a3e878ca24f9793a0708611862830 |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-10T20:26:47Z |
| publishDate | 2020-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-130a3e878ca24f9793a07086118628302023-11-19T21:42:59ZengMDPI AGEnergies1996-10732020-04-01138195010.3390/en13081950Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing SchemeJaehyun Lee0Eunjung Lee1Jinho Kim2School of Integrated Technology, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, KoreaSchool of Integrated Technology, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, KoreaSchool of Integrated Technology, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, KoreaIn the smart grid environment, the penetration of electric vehicle (EV) is increasing, and dynamic pricing and vehicle-to-grid technologies are being introduced. Consequently, automatic charging and discharging scheduling responding to electricity prices that change over time is required to reduce the charging cost of EVs, while increasing the grid reliability by moving charging loads from on-peak to off-peak periods. Hence, this study proposes a deep reinforcement learning-based, real-time EV charging and discharging algorithm. The proposed method utilizes kernel density estimation, particularly the nonparametric density function estimation method, to model the usage pattern of a specific charger at a specific location. Subsequently, the estimated density function is used to sample variables related to charger usage pattern so that the variables can be cast in the training process of a reinforcement learning agent. This ensures that the agent optimally learns the characteristics of the target charger. We analyzed the effectiveness of the proposed algorithm from two perspectives, i.e., charging cost and load shifting effect. Simulation results show that the proposed method outperforms the benchmarks that simply model usage pattern through general assumptions in terms of charging cost and load shifting effect. This means that when a reinforcement learning-based charging/discharging algorithm is deployed in a specific location, it is better to use data-driven approach to reflect the characteristics of the location, so that the charging cost reduction and the effect of load flattening are obtained.https://www.mdpi.com/1996-1073/13/8/1950electric vehicle charging/dischargingdynamic pricingreinforcement learningkernel density estimationcharger usage pattern |
| spellingShingle | Jaehyun Lee Eunjung Lee Jinho Kim Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme Energies electric vehicle charging/discharging dynamic pricing reinforcement learning kernel density estimation charger usage pattern |
| title | Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme |
| title_full | Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme |
| title_fullStr | Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme |
| title_full_unstemmed | Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme |
| title_short | Electric Vehicle Charging and Discharging Algorithm Based on Reinforcement Learning with Data-Driven Approach in Dynamic Pricing Scheme |
| title_sort | electric vehicle charging and discharging algorithm based on reinforcement learning with data driven approach in dynamic pricing scheme |
| topic | electric vehicle charging/discharging dynamic pricing reinforcement learning kernel density estimation charger usage pattern |
| url | https://www.mdpi.com/1996-1073/13/8/1950 |
| work_keys_str_mv | AT jaehyunlee electricvehiclecharginganddischargingalgorithmbasedonreinforcementlearningwithdatadrivenapproachindynamicpricingscheme AT eunjunglee electricvehiclecharginganddischargingalgorithmbasedonreinforcementlearningwithdatadrivenapproachindynamicpricingscheme AT jinhokim electricvehiclecharginganddischargingalgorithmbasedonreinforcementlearningwithdatadrivenapproachindynamicpricingscheme |