Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning
Heating, ventilation, and air conditioning (HVAC) energy consumption now accounts for a major portion of energy use for buildings. Therefore, finding the optimal energy-saving control strategy for HVAC systems to optimize energy consumption has become crucial in realizing energy savings, emission re...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10220078/ |
| _version_ | 1797736953052921856 |
|---|---|
| author | Yi Peng Haojun Shen Xiaochang Tang Sizhe Zhang Jinxiao Zhao Yuru Liu Yuming Nie |
| author_facet | Yi Peng Haojun Shen Xiaochang Tang Sizhe Zhang Jinxiao Zhao Yuru Liu Yuming Nie |
| author_sort | Yi Peng |
| collection | DOAJ |
| description | Heating, ventilation, and air conditioning (HVAC) energy consumption now accounts for a major portion of energy use for buildings. Therefore, finding the optimal energy-saving control strategy for HVAC systems to optimize energy consumption has become crucial in realizing energy savings, emission reductions, and green buildings. Traditional methods for HVAC parameter control require complex physical model calculation; continuous and coupled parameters are handled poorly. Developing deep reinforcement learning (DRL) methods provides new ideas for HVAC energy consumption optimization. Herein, a DRL-based energy consumption optimization framework for HVAC systems is proposed. First, an HVAC system energy consumption prediction model based on a convolutional neural network–long short-term memory (CNN-LSTM) network is suggested to approximate the real world. This model solves the efficiency problem of energy consumption prediction while also providing highly accurate predictions of HVAC energy consumption. We propose an enhanced deep deterministic policy gradient (E-DDPG) energy consumption optimization algorithm for HVAC systems based on an improved training strategy to obtain the best real-time energy consumption control strategy for HVAC systems. Finally, experiments using real-world building HVAC control data sets were conducted to evaluate our models. The experiments show that the CNN–LSTM model for HVAC system energy consumption prediction outperforms baseline models while reducing training time by 42.9%. Compared to the baseline algorithm, the E-DDPG algorithm using an improved training strategy requires 20% fewer iterations for convergence, has a 14.8% narrower fluctuation interval during the training process, and improves the energy efficiency ratio of HVAC systems by 49%. |
| first_indexed | 2024-03-12T13:21:21Z |
| format | Article |
| id | doaj.art-5dabc2160f6b48e18a7fe837a2716f68 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T13:21:21Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5dabc2160f6b48e18a7fe837a2716f682023-08-25T23:01:07ZengIEEEIEEE Access2169-35362023-01-0111882658827710.1109/ACCESS.2023.330568310220078Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement LearningYi Peng0https://orcid.org/0000-0003-2476-5908Haojun Shen1https://orcid.org/0009-0003-7962-7913Xiaochang Tang2Sizhe Zhang3Jinxiao Zhao4Yuru Liu5Yuming Nie6College of Oceanography and Space Informatics, China University of Petroleum, Qingdao, ChinaCollege of Oceanography and Space Informatics, China University of Petroleum, Qingdao, ChinaCollege of Computer Science and Technology, China University of Petroleum, Qingdao, ChinaCollege of Computer Science and Technology, China University of Petroleum, Qingdao, ChinaCollege of Computer Science and Technology, China University of Petroleum, Qingdao, ChinaCollege of Computer Science and Technology, China University of Petroleum, Qingdao, ChinaCollege of Computer Science and Technology, China University of Petroleum, Qingdao, ChinaHeating, ventilation, and air conditioning (HVAC) energy consumption now accounts for a major portion of energy use for buildings. Therefore, finding the optimal energy-saving control strategy for HVAC systems to optimize energy consumption has become crucial in realizing energy savings, emission reductions, and green buildings. Traditional methods for HVAC parameter control require complex physical model calculation; continuous and coupled parameters are handled poorly. Developing deep reinforcement learning (DRL) methods provides new ideas for HVAC energy consumption optimization. Herein, a DRL-based energy consumption optimization framework for HVAC systems is proposed. First, an HVAC system energy consumption prediction model based on a convolutional neural network–long short-term memory (CNN-LSTM) network is suggested to approximate the real world. This model solves the efficiency problem of energy consumption prediction while also providing highly accurate predictions of HVAC energy consumption. We propose an enhanced deep deterministic policy gradient (E-DDPG) energy consumption optimization algorithm for HVAC systems based on an improved training strategy to obtain the best real-time energy consumption control strategy for HVAC systems. Finally, experiments using real-world building HVAC control data sets were conducted to evaluate our models. The experiments show that the CNN–LSTM model for HVAC system energy consumption prediction outperforms baseline models while reducing training time by 42.9%. Compared to the baseline algorithm, the E-DDPG algorithm using an improved training strategy requires 20% fewer iterations for convergence, has a 14.8% narrower fluctuation interval during the training process, and improves the energy efficiency ratio of HVAC systems by 49%.https://ieeexplore.ieee.org/document/10220078/HVAC systemsDDPGdeep reinforcement learningenergy consumption |
| spellingShingle | Yi Peng Haojun Shen Xiaochang Tang Sizhe Zhang Jinxiao Zhao Yuru Liu Yuming Nie Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning IEEE Access HVAC systems DDPG deep reinforcement learning energy consumption |
| title | Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning |
| title_full | Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning |
| title_fullStr | Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning |
| title_full_unstemmed | Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning |
| title_short | Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning |
| title_sort | energy consumption optimization for heating ventilation and air conditioning systems based on deep reinforcement learning |
| topic | HVAC systems DDPG deep reinforcement learning energy consumption |
| url | https://ieeexplore.ieee.org/document/10220078/ |
| work_keys_str_mv | AT yipeng energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT haojunshen energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT xiaochangtang energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT sizhezhang energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT jinxiaozhao energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT yuruliu energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning AT yumingnie energyconsumptionoptimizationforheatingventilationandairconditioningsystemsbasedondeepreinforcementlearning |