Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading
Peer-to-peer (P2P) energy trading system has the ability to completely revolutionize the current household energy system by sharing energy among residents. As the number of customers employing distributed energy resources (DERs) such as solar rooftops increase, innovation in the double auction marke...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9963562/ |
| _version_ | 1811177799909638144 |
|---|---|
| author | Manassakan Sanayha Peerapon Vateekul |
| author_facet | Manassakan Sanayha Peerapon Vateekul |
| author_sort | Manassakan Sanayha |
| collection | DOAJ |
| description | Peer-to-peer (P2P) energy trading system has the ability to completely revolutionize the current household energy system by sharing energy among residents. As the number of customers employing distributed energy resources (DERs) such as solar rooftops increase, innovation in the double auction market (DA) system is becoming more significant. In this paper, a novel model-based, multi-agent asynchronous advantage actor-centralized-critic with communication (MB-A3C3) approach is carried out. Previous studies are limited since they suffer from unpredictable behavior in renewable energy resources and a large number of prosumers in the peer-to-peer market. As for the model-based strategy, we forecast the trading price and trading quantity in the daily energy trading system in order to overcome unpredictable issues. For the large number of prosumers, the multi-agent and multithreading RL has been chosen as our backbone since the prosumers’ behavior can be diverse; time-series clustering is introduced based on their daily trading behavior. With its environmental model and multi-threaded mechanism, MB-A3C3 is seen to be most efficient in carrying out tasks regards time and precision. The model is conducted on a large scale real-world hourly 2012–2013 dataset of 300 households in Sydney having rooftop solar systems installed in New South Wales (NSW), Australia. Results reveal that the MB-A3C3 approach outperforms other reinforcement learning methods (MADDPG and A3C3), producing lower community energy bills for 300 households. When internal trade (trading among houses) increased and external trade (trading to the grid) decreased, our multiple agent RL (MB-A3C3) significantly lowered energy bills by 17%. In closing the gap between the real-world and theoretical problems, the algorithms herein aid in reducing customers’ electricity bills. |
| first_indexed | 2024-04-11T06:08:25Z |
| format | Article |
| id | doaj.art-3821b8d85ca04983b1f0265223ccda73 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T06:08:25Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-3821b8d85ca04983b1f0265223ccda732022-12-22T04:41:25ZengIEEEIEEE Access2169-35362022-01-011012788212789310.1109/ACCESS.2022.32244609963562Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy TradingManassakan Sanayha0https://orcid.org/0000-0001-5402-839XPeerapon Vateekul1https://orcid.org/0000-0001-9718-3592Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Pathumwan, Bangkok, ThailandDepartment of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Pathumwan, Bangkok, ThailandPeer-to-peer (P2P) energy trading system has the ability to completely revolutionize the current household energy system by sharing energy among residents. As the number of customers employing distributed energy resources (DERs) such as solar rooftops increase, innovation in the double auction market (DA) system is becoming more significant. In this paper, a novel model-based, multi-agent asynchronous advantage actor-centralized-critic with communication (MB-A3C3) approach is carried out. Previous studies are limited since they suffer from unpredictable behavior in renewable energy resources and a large number of prosumers in the peer-to-peer market. As for the model-based strategy, we forecast the trading price and trading quantity in the daily energy trading system in order to overcome unpredictable issues. For the large number of prosumers, the multi-agent and multithreading RL has been chosen as our backbone since the prosumers’ behavior can be diverse; time-series clustering is introduced based on their daily trading behavior. With its environmental model and multi-threaded mechanism, MB-A3C3 is seen to be most efficient in carrying out tasks regards time and precision. The model is conducted on a large scale real-world hourly 2012–2013 dataset of 300 households in Sydney having rooftop solar systems installed in New South Wales (NSW), Australia. Results reveal that the MB-A3C3 approach outperforms other reinforcement learning methods (MADDPG and A3C3), producing lower community energy bills for 300 households. When internal trade (trading among houses) increased and external trade (trading to the grid) decreased, our multiple agent RL (MB-A3C3) significantly lowered energy bills by 17%. In closing the gap between the real-world and theoretical problems, the algorithms herein aid in reducing customers’ electricity bills.https://ieeexplore.ieee.org/document/9963562/Peer-to-peer energy tradingmodel-based reinforcement learningmulti-agent reinforcement learningdeep learning approach |
| spellingShingle | Manassakan Sanayha Peerapon Vateekul Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading IEEE Access Peer-to-peer energy trading model-based reinforcement learning multi-agent reinforcement learning deep learning approach |
| title | Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading |
| title_full | Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading |
| title_fullStr | Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading |
| title_full_unstemmed | Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading |
| title_short | Model-Based Approach on Multi-Agent Deep Reinforcement Learning With Multiple Clusters for Peer-To-Peer Energy Trading |
| title_sort | model based approach on multi agent deep reinforcement learning with multiple clusters for peer to peer energy trading |
| topic | Peer-to-peer energy trading model-based reinforcement learning multi-agent reinforcement learning deep learning approach |
| url | https://ieeexplore.ieee.org/document/9963562/ |
| work_keys_str_mv | AT manassakansanayha modelbasedapproachonmultiagentdeepreinforcementlearningwithmultipleclustersforpeertopeerenergytrading AT peeraponvateekul modelbasedapproachonmultiagentdeepreinforcementlearningwithmultipleclustersforpeertopeerenergytrading |