Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning
In an energy harvesting wireless sensor node (EHWSN), balance of energy harvested and consumption using dynamic energy management to achieve the goal of perpetual operation is one of the most important research topics. In this study, a novel fuzzy Q-learning (FQL)-based dynamic energy management (FQ...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/15/9/3117 |
| _version_ | 1827672751823912960 |
|---|---|
| author | Roy Chaoming Hsu Tzu-Hao Lin Po-Cheng Su |
| author_facet | Roy Chaoming Hsu Tzu-Hao Lin Po-Cheng Su |
| author_sort | Roy Chaoming Hsu |
| collection | DOAJ |
| description | In an energy harvesting wireless sensor node (EHWSN), balance of energy harvested and consumption using dynamic energy management to achieve the goal of perpetual operation is one of the most important research topics. In this study, a novel fuzzy Q-learning (FQL)-based dynamic energy management (FQLDEM) is proposed in adapting its policy to the time varying environment, regarding both the harvested energy and the energy consumption of the WSN. The FQLDEM applies Q-learning to train, evaluate, and update the fuzzy rule base and then uses the fuzzy inference system (FIS) for determining the working duty cycle of the sensor of the EHWSN. Through the interaction with the energy harvesting environment, the learning agent of the FQL will be able to find the appropriate fuzzy rules in adapting the working duty cycle for the goal of energy neutrality such that the objective of perpetual operation of the EHWSN can be achieved. Experimental results show that the FQLDEM can maintain the battery charge status at a higher level than other existing methods did, such as the reinforcement learning (RL) method and dynamic duty cycle adaption (DDCA), and achieve the perpetual operation of the EHWSN. Furthermore, experimental results for required on-demand sensing measurements exhibit that the FQLDEM method can be slowly upgraded to meet 65% of the service quality control requirements in the early stage, which outperforms the RL-based and DDCA methods. |
| first_indexed | 2024-03-10T04:12:41Z |
| format | Article |
| id | doaj.art-059a99d58f7c4bebb72aa78f0b9ef8e2 |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-03-10T04:12:41Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-059a99d58f7c4bebb72aa78f0b9ef8e22023-11-23T08:06:45ZengMDPI AGEnergies1996-10732022-04-01159311710.3390/en15093117Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-LearningRoy Chaoming Hsu0Tzu-Hao Lin1Po-Cheng Su2Electrical Engineering Department, National Chiayi University, Chiayi City 600355, TaiwanElectrical Engineering Department, National Chiayi University, Chiayi City 600355, TaiwanElectrical Engineering Department, National Chiayi University, Chiayi City 600355, TaiwanIn an energy harvesting wireless sensor node (EHWSN), balance of energy harvested and consumption using dynamic energy management to achieve the goal of perpetual operation is one of the most important research topics. In this study, a novel fuzzy Q-learning (FQL)-based dynamic energy management (FQLDEM) is proposed in adapting its policy to the time varying environment, regarding both the harvested energy and the energy consumption of the WSN. The FQLDEM applies Q-learning to train, evaluate, and update the fuzzy rule base and then uses the fuzzy inference system (FIS) for determining the working duty cycle of the sensor of the EHWSN. Through the interaction with the energy harvesting environment, the learning agent of the FQL will be able to find the appropriate fuzzy rules in adapting the working duty cycle for the goal of energy neutrality such that the objective of perpetual operation of the EHWSN can be achieved. Experimental results show that the FQLDEM can maintain the battery charge status at a higher level than other existing methods did, such as the reinforcement learning (RL) method and dynamic duty cycle adaption (DDCA), and achieve the perpetual operation of the EHWSN. Furthermore, experimental results for required on-demand sensing measurements exhibit that the FQLDEM method can be slowly upgraded to meet 65% of the service quality control requirements in the early stage, which outperforms the RL-based and DDCA methods.https://www.mdpi.com/1996-1073/15/9/3117energy harvesting wireless sensor nodedynamic energy managementfuzzy Q-learningenergy neutralityperpetual operation |
| spellingShingle | Roy Chaoming Hsu Tzu-Hao Lin Po-Cheng Su Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning Energies energy harvesting wireless sensor node dynamic energy management fuzzy Q-learning energy neutrality perpetual operation |
| title | Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning |
| title_full | Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning |
| title_fullStr | Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning |
| title_full_unstemmed | Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning |
| title_short | Dynamic Energy Management for Perpetual Operation of Energy Harvesting Wireless Sensor Node Using Fuzzy Q-Learning |
| title_sort | dynamic energy management for perpetual operation of energy harvesting wireless sensor node using fuzzy q learning |
| topic | energy harvesting wireless sensor node dynamic energy management fuzzy Q-learning energy neutrality perpetual operation |
| url | https://www.mdpi.com/1996-1073/15/9/3117 |
| work_keys_str_mv | AT roychaominghsu dynamicenergymanagementforperpetualoperationofenergyharvestingwirelesssensornodeusingfuzzyqlearning AT tzuhaolin dynamicenergymanagementforperpetualoperationofenergyharvestingwirelesssensornodeusingfuzzyqlearning AT pochengsu dynamicenergymanagementforperpetualoperationofenergyharvestingwirelesssensornodeusingfuzzyqlearning |