Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid
Uncertainties of solar PhotoVoltaic (solar PV) generation and Electric Vehicle (EV) demand are major issues for Optimal Energy Management (OEM) tasks in MicroGrid (MG), especially regarding power system stability and increased overall demand. A Probabilistic Load Flow (PLF) with a Battery Energy Sto...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-10-01
|
| Series: | Energy Reports |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352484723008739 |
| _version_ | 1797388828837675008 |
|---|---|
| author | Niphon Kaewdornhan Rongrit Chatthaworn |
| author_facet | Niphon Kaewdornhan Rongrit Chatthaworn |
| author_sort | Niphon Kaewdornhan |
| collection | DOAJ |
| description | Uncertainties of solar PhotoVoltaic (solar PV) generation and Electric Vehicle (EV) demand are major issues for Optimal Energy Management (OEM) tasks in MicroGrid (MG), especially regarding power system stability and increased overall demand. A Probabilistic Load Flow (PLF) with a Battery Energy Storage System (BESS) controlled by an Energy Management System (EMS) can deal with these issues. However, the PLF and optimization algorithm based on the iterative method leads to a high computation burden. Therefore, this paper proposes a model-free data-driven approach assisted Deep Reinforcement Learning (DRL) to decrease the computation burden of PLF and problem-solving. Deep Neural Networks (DNNs) are developed as a model-free data-driven to estimate the power flow parameters of MG instead of PLF. Moreover, the DRL named a Deep Deterministic Policy Gradient (DDPG) is deployed as the optimization algorithm to find the optimal solution in the OEM task. In addition, finding appropriate parameters of the DDPG is proposed in this paper. To showcase the efficacy of the proposed method, a low-voltage distribution network is developed as the MG. Objective functions including exchanged energy, carbon emission, and BESS degradation costs are considered in this work. Simulation results indicate that the proposed method is capable of reducing the computation burden by 88.47% in comparison to the Differential Evolution (DE) algorithm that uses 10 populations. Moreover, the best agent model obtained from finding suitable parameters of DDPG can provide a total cost of 115.63 USD/day, which is the lowest cost compared with the cost obtained by the DE. |
| first_indexed | 2024-03-08T22:46:29Z |
| format | Article |
| id | doaj.art-97f09ffee22143f9a343f26bcd76c33e |
| institution | Directory Open Access Journal |
| issn | 2352-4847 |
| language | English |
| last_indexed | 2024-03-08T22:46:29Z |
| publishDate | 2023-10-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Energy Reports |
| spelling | doaj.art-97f09ffee22143f9a343f26bcd76c33e2023-12-17T06:38:58ZengElsevierEnergy Reports2352-48472023-10-019850858Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGridNiphon Kaewdornhan0Rongrit Chatthaworn1Department of Electrical Engineering, Faculty of Engineering, Khon Kaen University, Khon Kaen 40002, ThailandCorresponding author.; Department of Electrical Engineering, Faculty of Engineering, Khon Kaen University, Khon Kaen 40002, ThailandUncertainties of solar PhotoVoltaic (solar PV) generation and Electric Vehicle (EV) demand are major issues for Optimal Energy Management (OEM) tasks in MicroGrid (MG), especially regarding power system stability and increased overall demand. A Probabilistic Load Flow (PLF) with a Battery Energy Storage System (BESS) controlled by an Energy Management System (EMS) can deal with these issues. However, the PLF and optimization algorithm based on the iterative method leads to a high computation burden. Therefore, this paper proposes a model-free data-driven approach assisted Deep Reinforcement Learning (DRL) to decrease the computation burden of PLF and problem-solving. Deep Neural Networks (DNNs) are developed as a model-free data-driven to estimate the power flow parameters of MG instead of PLF. Moreover, the DRL named a Deep Deterministic Policy Gradient (DDPG) is deployed as the optimization algorithm to find the optimal solution in the OEM task. In addition, finding appropriate parameters of the DDPG is proposed in this paper. To showcase the efficacy of the proposed method, a low-voltage distribution network is developed as the MG. Objective functions including exchanged energy, carbon emission, and BESS degradation costs are considered in this work. Simulation results indicate that the proposed method is capable of reducing the computation burden by 88.47% in comparison to the Differential Evolution (DE) algorithm that uses 10 populations. Moreover, the best agent model obtained from finding suitable parameters of DDPG can provide a total cost of 115.63 USD/day, which is the lowest cost compared with the cost obtained by the DE.http://www.sciencedirect.com/science/article/pii/S2352484723008739Deep neural networkDeep reinforcement learningElectric vehicleMicrogridPhotovoltaic generationProbabilistic load flow |
| spellingShingle | Niphon Kaewdornhan Rongrit Chatthaworn Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid Energy Reports Deep neural network Deep reinforcement learning Electric vehicle Microgrid Photovoltaic generation Probabilistic load flow |
| title | Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid |
| title_full | Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid |
| title_fullStr | Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid |
| title_full_unstemmed | Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid |
| title_short | Model-free data-driven approach assisted Deep Reinforcement Learning for Optimal Energy Management in MicroGrid |
| title_sort | model free data driven approach assisted deep reinforcement learning for optimal energy management in microgrid |
| topic | Deep neural network Deep reinforcement learning Electric vehicle Microgrid Photovoltaic generation Probabilistic load flow |
| url | http://www.sciencedirect.com/science/article/pii/S2352484723008739 |
| work_keys_str_mv | AT niphonkaewdornhan modelfreedatadrivenapproachassisteddeepreinforcementlearningforoptimalenergymanagementinmicrogrid AT rongritchatthaworn modelfreedatadrivenapproachassisteddeepreinforcementlearningforoptimalenergymanagementinmicrogrid |