Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty
Currently, renewable-energy-based power generation is rapidly developing to tackle climate change; however, the use of renewable energy is limited owing to the uncertainty related to renewable energy sources. In particular, energy storage systems (ESSs), which are critical for implementing wind powe...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8967100/ |
| _version_ | 1811204450532982784 |
|---|---|
| author | Eunsung Oh Hanho Wang |
| author_facet | Eunsung Oh Hanho Wang |
| author_sort | Eunsung Oh |
| collection | DOAJ |
| description | Currently, renewable-energy-based power generation is rapidly developing to tackle climate change; however, the use of renewable energy is limited owing to the uncertainty related to renewable energy sources. In particular, energy storage systems (ESSs), which are critical for implementing wind power generation (WPG), entail a wide uncertainty range. Herein, a reinforcement leaning (RL)-based ESS operation strategy is investigated for managing the WPG forecast uncertainty. First, a WPG forecast uncertainty minimization problem is formulated with respect to the ESS operation, subject to ESS constraints, and then, the problem is presented as a Markov decision process (MDP) model, with the state-action space limited by the ESS characteristics. To achieve the optimal solution of the MDP model, an expected state–action–reward–state–action (SARSA) method, which is robust toward the dispersion of the system environment, is employed. Further, frequency-domain data screening based on the k-mean clustering method is implemented to improve learning performance by reducing the variance of the WPG forecast uncertainty. Extensive simulations are conducted based on practical WPG generation data and forecasting. Results indicate that the proposed clustered RL-based ESS operation strategy can manage the WPG forecast uncertainty more effectively than conventional Q-learning-based methods; additionally, the proposed method demonstrates a near-optimal performance within a 1%-point analysis gap to the optimal solution, which requires complete information, including future values. |
| first_indexed | 2024-04-12T03:13:21Z |
| format | Article |
| id | doaj.art-9cd4a30d3d4b490ea62f9450faf385f1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T03:13:21Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-9cd4a30d3d4b490ea62f9450faf385f12022-12-22T03:50:15ZengIEEEIEEE Access2169-35362020-01-018209652097610.1109/ACCESS.2020.29688418967100Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast UncertaintyEunsung Oh0https://orcid.org/0000-0002-3161-8255Hanho Wang1https://orcid.org/0000-0002-0672-7775Department of Electrical and Electronic Engineering, Hanseo University, Chungcheongnam, South KoreaDepartment Smart Information and Telecommunication Engineering, Sangmyung University, Chungcheongnam, South KoreaCurrently, renewable-energy-based power generation is rapidly developing to tackle climate change; however, the use of renewable energy is limited owing to the uncertainty related to renewable energy sources. In particular, energy storage systems (ESSs), which are critical for implementing wind power generation (WPG), entail a wide uncertainty range. Herein, a reinforcement leaning (RL)-based ESS operation strategy is investigated for managing the WPG forecast uncertainty. First, a WPG forecast uncertainty minimization problem is formulated with respect to the ESS operation, subject to ESS constraints, and then, the problem is presented as a Markov decision process (MDP) model, with the state-action space limited by the ESS characteristics. To achieve the optimal solution of the MDP model, an expected state–action–reward–state–action (SARSA) method, which is robust toward the dispersion of the system environment, is employed. Further, frequency-domain data screening based on the k-mean clustering method is implemented to improve learning performance by reducing the variance of the WPG forecast uncertainty. Extensive simulations are conducted based on practical WPG generation data and forecasting. Results indicate that the proposed clustered RL-based ESS operation strategy can manage the WPG forecast uncertainty more effectively than conventional Q-learning-based methods; additionally, the proposed method demonstrates a near-optimal performance within a 1%-point analysis gap to the optimal solution, which requires complete information, including future values.https://ieeexplore.ieee.org/document/8967100/Energy storageforecastingMarkov decision processmean absolute errorreinforcement learningreliability |
| spellingShingle | Eunsung Oh Hanho Wang Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty IEEE Access Energy storage forecasting Markov decision process mean absolute error reinforcement learning reliability |
| title | Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty |
| title_full | Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty |
| title_fullStr | Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty |
| title_full_unstemmed | Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty |
| title_short | Reinforcement-Learning-Based Energy Storage System Operation Strategies to Manage Wind Power Forecast Uncertainty |
| title_sort | reinforcement learning based energy storage system operation strategies to manage wind power forecast uncertainty |
| topic | Energy storage forecasting Markov decision process mean absolute error reinforcement learning reliability |
| url | https://ieeexplore.ieee.org/document/8967100/ |
| work_keys_str_mv | AT eunsungoh reinforcementlearningbasedenergystoragesystemoperationstrategiestomanagewindpowerforecastuncertainty AT hanhowang reinforcementlearningbasedenergystoragesystemoperationstrategiestomanagewindpowerforecastuncertainty |