Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach
Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multi-objective optimization problem, involving targets such as maximizing market compensation and minimizing...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/15/14/4960 |
| _version_ | 1797406662331465728 |
|---|---|
| author | Harri Aaltonen Seppo Sierla Ville Kyrki Mahdi Pourakbari-Kasmaei Valeriy Vyatkin |
| author_facet | Harri Aaltonen Seppo Sierla Ville Kyrki Mahdi Pourakbari-Kasmaei Valeriy Vyatkin |
| author_sort | Harri Aaltonen |
| collection | DOAJ |
| description | Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multi-objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physics-based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs. |
| first_indexed | 2024-03-09T03:29:54Z |
| format | Article |
| id | doaj.art-c5202d3f5a9145568e73622918e0174b |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-09T03:29:54Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-c5202d3f5a9145568e73622918e0174b2023-12-03T14:58:00ZengMDPI AGEnergies1996-10732022-07-011514496010.3390/en15144960Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning ApproachHarri Aaltonen0Seppo Sierla1Ville Kyrki2Mahdi Pourakbari-Kasmaei3Valeriy Vyatkin4Department of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, FinlandDepartment of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, FinlandDepartment of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, FinlandDepartment of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, FinlandDepartment of Electrical Engineering and Automation, School of Electrical Engineering, Aalto University, FI-00076 Espoo, FinlandBattery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multi-objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physics-based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.https://www.mdpi.com/1996-1073/15/14/4960battery storagereinforcement learningmachine learningprimary frequency reservefrequency containment reservesimulation |
| spellingShingle | Harri Aaltonen Seppo Sierla Ville Kyrki Mahdi Pourakbari-Kasmaei Valeriy Vyatkin Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach Energies battery storage reinforcement learning machine learning primary frequency reserve frequency containment reserve simulation |
| title | Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach |
| title_full | Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach |
| title_fullStr | Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach |
| title_full_unstemmed | Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach |
| title_short | Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach |
| title_sort | bidding a battery on electricity markets and minimizing battery aging costs a reinforcement learning approach |
| topic | battery storage reinforcement learning machine learning primary frequency reserve frequency containment reserve simulation |
| url | https://www.mdpi.com/1996-1073/15/14/4960 |
| work_keys_str_mv | AT harriaaltonen biddingabatteryonelectricitymarketsandminimizingbatteryagingcostsareinforcementlearningapproach AT sepposierla biddingabatteryonelectricitymarketsandminimizingbatteryagingcostsareinforcementlearningapproach AT villekyrki biddingabatteryonelectricitymarketsandminimizingbatteryagingcostsareinforcementlearningapproach AT mahdipourakbarikasmaei biddingabatteryonelectricitymarketsandminimizingbatteryagingcostsareinforcementlearningapproach AT valeriyvyatkin biddingabatteryonelectricitymarketsandminimizingbatteryagingcostsareinforcementlearningapproach |