Reinforcement Learning for Tackling Energy-Saving and Energy-Balance Dilemma of Cluster-Based Routing Protocols in WSNs - Yan Wang; Ke Deng; Yongli Ren; Jeffrey Chan; Guangjian Tian; Christian S. Jensen (IEEE Access, 2024)
In Wireless Sensor Networks (WSNs), where sensor batteries cannot be replaced or recharged easily, the limited availability of energy has motivated a variety of cluster-based routing protocols. The unit operation of these protocols is known as a round (aka. a cycle). A round has two phases: 1) sensor clustering for selecting cluster-heads, and 2) routing for transmitting sensed data from sensors to a base-station via cluster-heads in multi-hop. The cluster-based routing protocols face the dilemma of energy-saving and energy-balance in both phases. Simply speaking, we have to trade off energy-saving for energy-balance and vice versa. First, frequent clustering can improve the energy-balance by rotating cluster-heads often among sensors, but the overhead for clustering increases quickly. Second, the multi-hop routing saves the energy of cluster-heads far away from transmitting data directly to base-station, but it deteriorates energy-balance since the cluster-heads near the base-station suffer more energy consumption for transmitting data of their own clusters and data from other cluster-heads as relays. Even though the related issues have been discussed and investigated widely, no learning-based solution has been proposed yet. This study attempts to fill the gap. Instead of proposing a new cluster-based routing protocol, this study adapts reinforcement learning to make two decisions concurrently for the existing protocols: 1) whether to redo sensor clustering for each round and 2) which cluster-heads should be excluded from being relays in multi-hop routing for each round. With several tailored techniques, the reinforcement learning-based solution could significantly boost the existing protocols by improving energy-saving and energy-balance, as evidenced by extensive experiments.