Reinforcement learning-driven local transactive energy market for distributed energy resources
Local energy markets are emerging as a tool for coordinating generation, storage, and consumption of energy from distributed resources. In combination with automation, they promise to provide an effective energy management framework that is fair and brings system-level savings. The cooperative–compe...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-05-01
|
| Series: | Energy and AI |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666546822000118 |
| _version_ | 1818244873048817664 |
|---|---|
| author | Steven Zhang Daniel May Mustafa Gül Petr Musilek |
| author_facet | Steven Zhang Daniel May Mustafa Gül Petr Musilek |
| author_sort | Steven Zhang |
| collection | DOAJ |
| description | Local energy markets are emerging as a tool for coordinating generation, storage, and consumption of energy from distributed resources. In combination with automation, they promise to provide an effective energy management framework that is fair and brings system-level savings. The cooperative–competitive nature of energy markets calls for multi-agent based automation with learning energy trading agents. However, depending on the dynamics of the agent–environment interaction, this approach may yield unintended behavior of market participants. Thus, the design of market mechanisms suitable for reinforcement learning agents must take into account this interplay. This article introduces autonomous local energy exchange (ALEX) as an experimental framework that combines multi-agent learning and double auction mechanism. Participants determine their internal price signals and make energy management decisions through market interactions, rather than relying on predetermined external price signals. The main contribution of this article is examination of compatibility between specific market elements and independent learning agents. Effects of different market properties are evaluated through simulation experiments, and the results are used for determine a suitable market design. The results show that market truthfulness maintains demand-response functionality, while weak budget balancing provides a strong reinforcement signal for the learning agents. The resulting agent behavior is compared with two baselines: net billing and time-of-use rates. The ALEX-based pricing is more responsive to fluctuations in the community net load compared to the time-of-use. The more accurate accounting of renewable energy usage reduced bills by a median 38.8% compared to net billing, confirming the ability to better facilitate demand response. |
| first_indexed | 2024-12-12T14:23:56Z |
| format | Article |
| id | doaj.art-408478d9110d4224878f12ff4b6c51be |
| institution | Directory Open Access Journal |
| issn | 2666-5468 |
| language | English |
| last_indexed | 2024-12-12T14:23:56Z |
| publishDate | 2022-05-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Energy and AI |
| spelling | doaj.art-408478d9110d4224878f12ff4b6c51be2022-12-22T00:21:45ZengElsevierEnergy and AI2666-54682022-05-018100150Reinforcement learning-driven local transactive energy market for distributed energy resourcesSteven Zhang0Daniel May1Mustafa Gül2Petr Musilek3Electrical and Computer Engineering, University of Alberta, CanadaElectrical and Computer Engineering, University of Alberta, CanadaCivil and Environmental Engineering, University of Alberta, CanadaElectrical and Computer Engineering, University of Alberta, Canada; Corresponding author.Local energy markets are emerging as a tool for coordinating generation, storage, and consumption of energy from distributed resources. In combination with automation, they promise to provide an effective energy management framework that is fair and brings system-level savings. The cooperative–competitive nature of energy markets calls for multi-agent based automation with learning energy trading agents. However, depending on the dynamics of the agent–environment interaction, this approach may yield unintended behavior of market participants. Thus, the design of market mechanisms suitable for reinforcement learning agents must take into account this interplay. This article introduces autonomous local energy exchange (ALEX) as an experimental framework that combines multi-agent learning and double auction mechanism. Participants determine their internal price signals and make energy management decisions through market interactions, rather than relying on predetermined external price signals. The main contribution of this article is examination of compatibility between specific market elements and independent learning agents. Effects of different market properties are evaluated through simulation experiments, and the results are used for determine a suitable market design. The results show that market truthfulness maintains demand-response functionality, while weak budget balancing provides a strong reinforcement signal for the learning agents. The resulting agent behavior is compared with two baselines: net billing and time-of-use rates. The ALEX-based pricing is more responsive to fluctuations in the community net load compared to the time-of-use. The more accurate accounting of renewable energy usage reduced bills by a median 38.8% compared to net billing, confirming the ability to better facilitate demand response.http://www.sciencedirect.com/science/article/pii/S2666546822000118Transactive energyDemand responseDistributed energy resources (DER)DER integrationLocal energy marketReinforcement learning |
| spellingShingle | Steven Zhang Daniel May Mustafa Gül Petr Musilek Reinforcement learning-driven local transactive energy market for distributed energy resources Energy and AI Transactive energy Demand response Distributed energy resources (DER) DER integration Local energy market Reinforcement learning |
| title | Reinforcement learning-driven local transactive energy market for distributed energy resources |
| title_full | Reinforcement learning-driven local transactive energy market for distributed energy resources |
| title_fullStr | Reinforcement learning-driven local transactive energy market for distributed energy resources |
| title_full_unstemmed | Reinforcement learning-driven local transactive energy market for distributed energy resources |
| title_short | Reinforcement learning-driven local transactive energy market for distributed energy resources |
| title_sort | reinforcement learning driven local transactive energy market for distributed energy resources |
| topic | Transactive energy Demand response Distributed energy resources (DER) DER integration Local energy market Reinforcement learning |
| url | http://www.sciencedirect.com/science/article/pii/S2666546822000118 |
| work_keys_str_mv | AT stevenzhang reinforcementlearningdrivenlocaltransactiveenergymarketfordistributedenergyresources AT danielmay reinforcementlearningdrivenlocaltransactiveenergymarketfordistributedenergyresources AT mustafagul reinforcementlearningdrivenlocaltransactiveenergymarketfordistributedenergyresources AT petrmusilek reinforcementlearningdrivenlocaltransactiveenergymarketfordistributedenergyresources |