Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction
In recent years, home energy management systems (HEMS), which enable the automatic control of electrical equipment and home appliances, have been attracting attention as a method for saving electricity at home. HEMS achieve energy saving by visualizing energy consumption at home and controlling ener...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9606721/ |
| _version_ | 1818837514400563200 |
|---|---|
| author | Naoki Kodama Taku Harada Kazuteru Miyazaki |
| author_facet | Naoki Kodama Taku Harada Kazuteru Miyazaki |
| author_sort | Naoki Kodama |
| collection | DOAJ |
| description | In recent years, home energy management systems (HEMS), which enable the automatic control of electrical equipment and home appliances, have been attracting attention as a method for saving electricity at home. HEMS achieve energy saving by visualizing energy consumption at home and controlling energy consuming equipment such as air conditioners. The optimum control law is difficult to attain, owing to uncertainties related to power demand and power supply from the electrical equipment. Deep reinforcement learning has been used to address energy optimization problems for home environments. However, in HEMS, several components such as heating, ventilation, and air conditioning (HVAC) systems, storage batteries, and electric water heaters are simultaneously controlled, and therefore, the action space becomes extremely large. Therefore, it may not be feasible to fully learn the rare experience using traditional deep reinforcement learning methods due to the large size of the state-action space and slow propagation of delayed rewards. In this study, we propose an energy management algorithm that uses the Dual Targeting Algorithm to strongly learn the experience of acquiring high returns using the quick propagation of delayed rewards via multistep returns. The proposed energy management algorithm is applied to a HEMS learning experiment to control a storage battery and an HVAC system, and its performance is compared to that of a Deep Deterministic Policy Gradient-based energy management system. As a result, it is confirmed that the proposed method can reduce the number of hours deviating from the comfort temperature range by about 17% compared to the existing method. |
| first_indexed | 2024-12-19T03:23:43Z |
| format | Article |
| id | doaj.art-22b2c6543c9a4030818574b00c440a55 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T03:23:43Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-22b2c6543c9a4030818574b00c440a552022-12-21T20:37:41ZengIEEEIEEE Access2169-35362021-01-01915310815311510.1109/ACCESS.2021.31263659606721Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep PredictionNaoki Kodama0https://orcid.org/0000-0003-3412-8558Taku Harada1https://orcid.org/0000-0001-5410-7514Kazuteru Miyazaki2https://orcid.org/0000-0001-8175-213XDepartment of Computer Science, School of Science and Technology, Meiji University, Kawasaki, Kanagawa, JapanDepartment of Industrial Administration, Faculty of Science and Technology, Tokyo University of Science, Noda-shi, Chiba, JapanNational Institution for Academic Degrees and Quality Enhancement of Higher Education, Kodaira-shi, Tokyo, JapanIn recent years, home energy management systems (HEMS), which enable the automatic control of electrical equipment and home appliances, have been attracting attention as a method for saving electricity at home. HEMS achieve energy saving by visualizing energy consumption at home and controlling energy consuming equipment such as air conditioners. The optimum control law is difficult to attain, owing to uncertainties related to power demand and power supply from the electrical equipment. Deep reinforcement learning has been used to address energy optimization problems for home environments. However, in HEMS, several components such as heating, ventilation, and air conditioning (HVAC) systems, storage batteries, and electric water heaters are simultaneously controlled, and therefore, the action space becomes extremely large. Therefore, it may not be feasible to fully learn the rare experience using traditional deep reinforcement learning methods due to the large size of the state-action space and slow propagation of delayed rewards. In this study, we propose an energy management algorithm that uses the Dual Targeting Algorithm to strongly learn the experience of acquiring high returns using the quick propagation of delayed rewards via multistep returns. The proposed energy management algorithm is applied to a HEMS learning experiment to control a storage battery and an HVAC system, and its performance is compared to that of a Deep Deterministic Policy Gradient-based energy management system. As a result, it is confirmed that the proposed method can reduce the number of hours deviating from the comfort temperature range by about 17% compared to the existing method.https://ieeexplore.ieee.org/document/9606721/Deep reinforcement learningdeep Q-networkQ-learningenergy managementenergy cost |
| spellingShingle | Naoki Kodama Taku Harada Kazuteru Miyazaki Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction IEEE Access Deep reinforcement learning deep Q-network Q-learning energy management energy cost |
| title | Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction |
| title_full | Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction |
| title_fullStr | Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction |
| title_full_unstemmed | Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction |
| title_short | Home Energy Management Algorithm Based on Deep Reinforcement Learning Using Multistep Prediction |
| title_sort | home energy management algorithm based on deep reinforcement learning using multistep prediction |
| topic | Deep reinforcement learning deep Q-network Q-learning energy management energy cost |
| url | https://ieeexplore.ieee.org/document/9606721/ |
| work_keys_str_mv | AT naokikodama homeenergymanagementalgorithmbasedondeepreinforcementlearningusingmultistepprediction AT takuharada homeenergymanagementalgorithmbasedondeepreinforcementlearningusingmultistepprediction AT kazuterumiyazaki homeenergymanagementalgorithmbasedondeepreinforcementlearningusingmultistepprediction |