A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids
Decision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this pa...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-07-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/12/15/2891 |
| _version_ | 1798002859679154176 |
|---|---|
| author | Ning Wang Weisheng Xu Weihui Shao Zhiyu Xu |
| author_facet | Ning Wang Weisheng Xu Weihui Shao Zhiyu Xu |
| author_sort | Ning Wang |
| collection | DOAJ |
| description | Decision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this paper, we investigate the potential of applying a Q-learning algorithm into a continuous double auction mechanism. By choosing a global supply and demand relationship as states and considering both bidding price and quantity as actions, a new Q-learning architecture is proposed to better reflect personalized bidding preferences and response to real-time market conditions. The application of battery energy storage system performs an alternative form of demand response by exerting potential capacity. A Q-cube framework is designed to describe the Q-value distribution iteration. Results from a case study on 14 microgrids in Guizhou Province, China indicate that the proposed Q-cube framework is capable of making rational bidding decisions and raising the microgrids’ profits. |
| first_indexed | 2024-04-11T11:58:54Z |
| format | Article |
| id | doaj.art-c13067ce146e473a8423f7d1ae8d227e |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-04-11T11:58:54Z |
| publishDate | 2019-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-c13067ce146e473a8423f7d1ae8d227e2022-12-22T04:24:57ZengMDPI AGEnergies1996-10732019-07-011215289110.3390/en12152891en12152891A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among MicrogridsNing Wang0Weisheng Xu1Weihui Shao2Zhiyu Xu3School of Electronics and Information Engineering, Tongji University, Shanghai 201804, ChinaSchool of Electronics and Information Engineering, Tongji University, Shanghai 201804, ChinaEducation Technology and Computing Center, Tongji University, Shanghai 200092, ChinaSchool of Electronics and Information Engineering, Tongji University, Shanghai 201804, ChinaDecision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this paper, we investigate the potential of applying a Q-learning algorithm into a continuous double auction mechanism. By choosing a global supply and demand relationship as states and considering both bidding price and quantity as actions, a new Q-learning architecture is proposed to better reflect personalized bidding preferences and response to real-time market conditions. The application of battery energy storage system performs an alternative form of demand response by exerting potential capacity. A Q-cube framework is designed to describe the Q-value distribution iteration. Results from a case study on 14 microgrids in Guizhou Province, China indicate that the proposed Q-cube framework is capable of making rational bidding decisions and raising the microgrids’ profits.https://www.mdpi.com/1996-1073/12/15/2891microgridscontinuous double auctionQ-learning algorithmbattery energy storage system, Q-cube frameworkbidding strategy |
| spellingShingle | Ning Wang Weisheng Xu Weihui Shao Zhiyu Xu A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids Energies microgrids continuous double auction Q-learning algorithm battery energy storage system, Q-cube framework bidding strategy |
| title | A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids |
| title_full | A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids |
| title_fullStr | A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids |
| title_full_unstemmed | A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids |
| title_short | A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids |
| title_sort | q cube framework of reinforcement learning algorithm for continuous double auction among microgrids |
| topic | microgrids continuous double auction Q-learning algorithm battery energy storage system, Q-cube framework bidding strategy |
| url | https://www.mdpi.com/1996-1073/12/15/2891 |
| work_keys_str_mv | AT ningwang aqcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT weishengxu aqcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT weihuishao aqcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT zhiyuxu aqcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT ningwang qcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT weishengxu qcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT weihuishao qcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids AT zhiyuxu qcubeframeworkofreinforcementlearningalgorithmforcontinuousdoubleauctionamongmicrogrids |