A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks
Underwater sensor networks (UWSNs) have become a hot research topic because of their various aquatic applications. As the underwater sensor nodes are powered by built-in batteries which are difficult to replace, extending the network lifetime is a most urgent need. Due to the low and variable transm...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2017-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/17/7/1660 |
| _version_ | 1798005104659398656 |
|---|---|
| author | Zhigang Jin Yingying Ma Yishan Su Shuo Li Xiaomei Fu |
| author_facet | Zhigang Jin Yingying Ma Yishan Su Shuo Li Xiaomei Fu |
| author_sort | Zhigang Jin |
| collection | DOAJ |
| description | Underwater sensor networks (UWSNs) have become a hot research topic because of their various aquatic applications. As the underwater sensor nodes are powered by built-in batteries which are difficult to replace, extending the network lifetime is a most urgent need. Due to the low and variable transmission speed of sound, the design of reliable routing algorithms for UWSNs is challenging. In this paper, we propose a Q-learning based delay-aware routing (QDAR) algorithm to extend the lifetime of underwater sensor networks. In QDAR, a data collection phase is designed to adapt to the dynamic environment. With the application of the Q-learning technique, QDAR can determine a global optimal next hop rather than a greedy one. We define an action-utility function in which residual energy and propagation delay are both considered for adequate routing decisions. Thus, the QDAR algorithm can extend the network lifetime by uniformly distributing the residual energy and provide lower end-to-end delay. The simulation results show that our protocol can yield nearly the same network lifetime, and can reduce the end-to-end delay by 20–25% compared with a classic lifetime-extended routing protocol (QELAR). |
| first_indexed | 2024-04-11T12:35:01Z |
| format | Article |
| id | doaj.art-e90ca255f9254d7dbea780b7e3e08f4e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T12:35:01Z |
| publishDate | 2017-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-e90ca255f9254d7dbea780b7e3e08f4e2022-12-22T04:23:40ZengMDPI AGSensors1424-82202017-07-01177166010.3390/s17071660s17071660A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor NetworksZhigang Jin0Yingying Ma1Yishan Su2Shuo Li3Xiaomei Fu4School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Marine Science and Technology, Tianjin University, Tianjin 300072, ChinaUnderwater sensor networks (UWSNs) have become a hot research topic because of their various aquatic applications. As the underwater sensor nodes are powered by built-in batteries which are difficult to replace, extending the network lifetime is a most urgent need. Due to the low and variable transmission speed of sound, the design of reliable routing algorithms for UWSNs is challenging. In this paper, we propose a Q-learning based delay-aware routing (QDAR) algorithm to extend the lifetime of underwater sensor networks. In QDAR, a data collection phase is designed to adapt to the dynamic environment. With the application of the Q-learning technique, QDAR can determine a global optimal next hop rather than a greedy one. We define an action-utility function in which residual energy and propagation delay are both considered for adequate routing decisions. Thus, the QDAR algorithm can extend the network lifetime by uniformly distributing the residual energy and provide lower end-to-end delay. The simulation results show that our protocol can yield nearly the same network lifetime, and can reduce the end-to-end delay by 20–25% compared with a classic lifetime-extended routing protocol (QELAR).https://www.mdpi.com/1424-8220/17/7/1660underwater sensor networksrouting protocollifetime-extendeddelay-awareQ-learning technique |
| spellingShingle | Zhigang Jin Yingying Ma Yishan Su Shuo Li Xiaomei Fu A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks Sensors underwater sensor networks routing protocol lifetime-extended delay-aware Q-learning technique |
| title | A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks |
| title_full | A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks |
| title_fullStr | A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks |
| title_full_unstemmed | A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks |
| title_short | A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks |
| title_sort | q learning based delay aware routing algorithm to extend the lifetime of underwater sensor networks |
| topic | underwater sensor networks routing protocol lifetime-extended delay-aware Q-learning technique |
| url | https://www.mdpi.com/1424-8220/17/7/1660 |
| work_keys_str_mv | AT zhigangjin aqlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT yingyingma aqlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT yishansu aqlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT shuoli aqlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT xiaomeifu aqlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT zhigangjin qlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT yingyingma qlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT yishansu qlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT shuoli qlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks AT xiaomeifu qlearningbaseddelayawareroutingalgorithmtoextendthelifetimeofunderwatersensornetworks |