Finding individual strategies for storage units in electricity market models using deep reinforcement learning
Abstract Modeling energy storage units realistically is challenging as their decision-making is not governed by a marginal cost pricing strategy but relies on expected electricity prices. Existing electricity market models often use centralized rule-based bidding or global optimization approaches, w...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-10-01
|
| Series: | Energy Informatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s42162-023-00293-0 |
| _version_ | 1827787200550404096 |
|---|---|
| author | Nick Harder Anke Weidlich Philipp Staudt |
| author_facet | Nick Harder Anke Weidlich Philipp Staudt |
| author_sort | Nick Harder |
| collection | DOAJ |
| description | Abstract Modeling energy storage units realistically is challenging as their decision-making is not governed by a marginal cost pricing strategy but relies on expected electricity prices. Existing electricity market models often use centralized rule-based bidding or global optimization approaches, which may not accurately capture the competitive behavior of market participants. To address this issue, we present a novel method using multi-agent deep reinforcement learning to model individual strategies in electricity market models. We demonstrate the practical applicability of our approach using a detailed model of the German wholesale electricity market with a complete fleet of pumped hydro energy storage units represented as learning agents. We compare the results to widely used modeling approaches and demonstrate that the proposed method performs well and can accurately represent the competitive behavior of market participants. To understand the benefits of using reinforcement learning, we analyze overall profits, aggregated dispatch, and individual behavior of energy storage units. The proposed method can improve the accuracy and realism of electricity market modeling and help policymakers make informed decisions for future market designs and policies. |
| first_indexed | 2024-03-11T16:46:04Z |
| format | Article |
| id | doaj.art-3768e654ad444f5a83530b0776125e1c |
| institution | Directory Open Access Journal |
| issn | 2520-8942 |
| language | English |
| last_indexed | 2024-03-11T16:46:04Z |
| publishDate | 2023-10-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Energy Informatics |
| spelling | doaj.art-3768e654ad444f5a83530b0776125e1c2023-10-22T11:28:51ZengSpringerOpenEnergy Informatics2520-89422023-10-016S112110.1186/s42162-023-00293-0Finding individual strategies for storage units in electricity market models using deep reinforcement learningNick Harder0Anke Weidlich1Philipp Staudt2Institute for Sustainable Systems Engineering, University of FreiburgInstitute for Sustainable Systems Engineering, University of FreiburgDepartment of Computing Science, University of OldenburgAbstract Modeling energy storage units realistically is challenging as their decision-making is not governed by a marginal cost pricing strategy but relies on expected electricity prices. Existing electricity market models often use centralized rule-based bidding or global optimization approaches, which may not accurately capture the competitive behavior of market participants. To address this issue, we present a novel method using multi-agent deep reinforcement learning to model individual strategies in electricity market models. We demonstrate the practical applicability of our approach using a detailed model of the German wholesale electricity market with a complete fleet of pumped hydro energy storage units represented as learning agents. We compare the results to widely used modeling approaches and demonstrate that the proposed method performs well and can accurately represent the competitive behavior of market participants. To understand the benefits of using reinforcement learning, we analyze overall profits, aggregated dispatch, and individual behavior of energy storage units. The proposed method can improve the accuracy and realism of electricity market modeling and help policymakers make informed decisions for future market designs and policies.https://doi.org/10.1186/s42162-023-00293-0Agent-based modelingElectricity marketsEnergy storageMulti-agent reinforcement learninReinforcement learning |
| spellingShingle | Nick Harder Anke Weidlich Philipp Staudt Finding individual strategies for storage units in electricity market models using deep reinforcement learning Energy Informatics Agent-based modeling Electricity markets Energy storage Multi-agent reinforcement learnin Reinforcement learning |
| title | Finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| title_full | Finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| title_fullStr | Finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| title_full_unstemmed | Finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| title_short | Finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| title_sort | finding individual strategies for storage units in electricity market models using deep reinforcement learning |
| topic | Agent-based modeling Electricity markets Energy storage Multi-agent reinforcement learnin Reinforcement learning |
| url | https://doi.org/10.1186/s42162-023-00293-0 |
| work_keys_str_mv | AT nickharder findingindividualstrategiesforstorageunitsinelectricitymarketmodelsusingdeepreinforcementlearning AT ankeweidlich findingindividualstrategiesforstorageunitsinelectricitymarketmodelsusingdeepreinforcementlearning AT philippstaudt findingindividualstrategiesforstorageunitsinelectricitymarketmodelsusingdeepreinforcementlearning |