Battery Energy Management in a Microgrid Using Batch Reinforcement Learning
Motivated by recent developments in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL in energy management in microgrids. We tackle the challenge of finding a closed-loop control policy to optimally schedule the operation of a storage device, in order to maximi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2017-11-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/10/11/1846 |
| _version_ | 1818014136997511168 |
|---|---|
| author | Brida V. Mbuwir Frederik Ruelens Fred Spiessens Geert Deconinck |
| author_facet | Brida V. Mbuwir Frederik Ruelens Fred Spiessens Geert Deconinck |
| author_sort | Brida V. Mbuwir |
| collection | DOAJ |
| description | Motivated by recent developments in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL in energy management in microgrids. We tackle the challenge of finding a closed-loop control policy to optimally schedule the operation of a storage device, in order to maximize self-consumption of local photovoltaic production in a microgrid. In this work, the fitted Q-iteration algorithm, a standard batch RL technique, is used by an RL agent to construct a control policy. The proposed method is data-driven and uses a state-action value function to find an optimal scheduling plan for a battery. The battery’s charge and discharge efficiencies, and the nonlinearity in the microgrid due to the inverter’s efficiency are taken into account. The proposed approach has been tested by simulation in a residential setting using data from Belgian residential consumers. The developed framework is benchmarked with a model-based technique, and the simulation results show a performance gap of 19%. The simulation results provide insight for developing optimal policies in more realistically-scaled and interconnected microgrids and for including uncertainties in generation and consumption for which white-box models become inaccurate and/or infeasible. |
| first_indexed | 2024-04-14T06:41:17Z |
| format | Article |
| id | doaj.art-b11ff664232841f586419359d580f100 |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-04-14T06:41:17Z |
| publishDate | 2017-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-b11ff664232841f586419359d580f1002022-12-22T02:07:19ZengMDPI AGEnergies1996-10732017-11-011011184610.3390/en10111846en10111846Battery Energy Management in a Microgrid Using Batch Reinforcement LearningBrida V. Mbuwir0Frederik Ruelens1Fred Spiessens2Geert Deconinck3ESAT/Electa, KU Leuven, Kasteelpark Arenberg 10 bus 2445, BE-3001 Leuven, BelgiumESAT/Electa, KU Leuven, Kasteelpark Arenberg 10 bus 2445, BE-3001 Leuven, BelgiumEnergy Department, EnergyVille, Thor Park, Poort Genk 8130, 3600 Genk, BelgiumESAT/Electa, KU Leuven, Kasteelpark Arenberg 10 bus 2445, BE-3001 Leuven, BelgiumMotivated by recent developments in batch Reinforcement Learning (RL), this paper contributes to the application of batch RL in energy management in microgrids. We tackle the challenge of finding a closed-loop control policy to optimally schedule the operation of a storage device, in order to maximize self-consumption of local photovoltaic production in a microgrid. In this work, the fitted Q-iteration algorithm, a standard batch RL technique, is used by an RL agent to construct a control policy. The proposed method is data-driven and uses a state-action value function to find an optimal scheduling plan for a battery. The battery’s charge and discharge efficiencies, and the nonlinearity in the microgrid due to the inverter’s efficiency are taken into account. The proposed approach has been tested by simulation in a residential setting using data from Belgian residential consumers. The developed framework is benchmarked with a model-based technique, and the simulation results show a performance gap of 19%. The simulation results provide insight for developing optimal policies in more realistically-scaled and interconnected microgrids and for including uncertainties in generation and consumption for which white-box models become inaccurate and/or infeasible.https://www.mdpi.com/1996-1073/10/11/1846control policyfitted-Q iterationmicrogridsreinforcement learning |
| spellingShingle | Brida V. Mbuwir Frederik Ruelens Fred Spiessens Geert Deconinck Battery Energy Management in a Microgrid Using Batch Reinforcement Learning Energies control policy fitted-Q iteration microgrids reinforcement learning |
| title | Battery Energy Management in a Microgrid Using Batch Reinforcement Learning |
| title_full | Battery Energy Management in a Microgrid Using Batch Reinforcement Learning |
| title_fullStr | Battery Energy Management in a Microgrid Using Batch Reinforcement Learning |
| title_full_unstemmed | Battery Energy Management in a Microgrid Using Batch Reinforcement Learning |
| title_short | Battery Energy Management in a Microgrid Using Batch Reinforcement Learning |
| title_sort | battery energy management in a microgrid using batch reinforcement learning |
| topic | control policy fitted-Q iteration microgrids reinforcement learning |
| url | https://www.mdpi.com/1996-1073/10/11/1846 |
| work_keys_str_mv | AT bridavmbuwir batteryenergymanagementinamicrogridusingbatchreinforcementlearning AT frederikruelens batteryenergymanagementinamicrogridusingbatchreinforcementlearning AT fredspiessens batteryenergymanagementinamicrogridusingbatchreinforcementlearning AT geertdeconinck batteryenergymanagementinamicrogridusingbatchreinforcementlearning |