DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks
Underwater acoustic sensor networks (UASNs) are challenged by the dynamic nature of the underwater environment, large propagation delays, and global positioning system (GPS) signal unavailability, which make traditional medium access control (MAC) protocols less effective. These factors limit the ch...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/9/4474 |
| _version_ | 1797601694402478080 |
|---|---|
| author | Slavica Tomovic Igor Radusinovic |
| author_facet | Slavica Tomovic Igor Radusinovic |
| author_sort | Slavica Tomovic |
| collection | DOAJ |
| description | Underwater acoustic sensor networks (UASNs) are challenged by the dynamic nature of the underwater environment, large propagation delays, and global positioning system (GPS) signal unavailability, which make traditional medium access control (MAC) protocols less effective. These factors limit the channel utilization and performance of UASNs, making it difficult to achieve high data rates and handle congestion. To address these challenges, we propose a reinforcement learning (RL) MAC protocol that supports asynchronous network operation and leverages large propagation delays to improve the network throughput.he protocol is based on framed ALOHA and enables nodes to learn an optimal transmission strategy in a fully distributed manner without requiring detailed information about the external environment. The transmission strategy of sensor nodes is defined as a combination of time-slot and transmission-offset selection. By relying on the concept of learning through interaction with the environment, the proposed protocol enhances network resilience and adaptability. In both static and mobile network scenarios, it has been compared with the state-of-the-art framed ALOHA for the underwater environment (UW-ALOHA-Q), carrier-sensing ALOHA (CS-ALOHA), and delay-aware opportunistic transmission scheduling (DOTS) protocols. The simulation results show that the proposed solution leads to significant channel utilization gains, ranging from 13% to 106% in static network scenarios and from 23% to 126% in mobile network scenarios.oreover, using a more efficient learning strategy, it significantly reduces convergence time compared to UW-ALOHA-Q in larger networks, despite the increased action space. |
| first_indexed | 2024-03-11T04:07:13Z |
| format | Article |
| id | doaj.art-ca33ade602944cd6898c4847b4cb3fa3 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T04:07:13Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-ca33ade602944cd6898c4847b4cb3fa32023-11-17T23:44:53ZengMDPI AGSensors1424-82202023-05-01239447410.3390/s23094474DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor NetworksSlavica Tomovic0Igor Radusinovic1Faculty of Electrical Engineering, University of Montenegro, 81000 Podgorica, MontenegroFaculty of Electrical Engineering, University of Montenegro, 81000 Podgorica, MontenegroUnderwater acoustic sensor networks (UASNs) are challenged by the dynamic nature of the underwater environment, large propagation delays, and global positioning system (GPS) signal unavailability, which make traditional medium access control (MAC) protocols less effective. These factors limit the channel utilization and performance of UASNs, making it difficult to achieve high data rates and handle congestion. To address these challenges, we propose a reinforcement learning (RL) MAC protocol that supports asynchronous network operation and leverages large propagation delays to improve the network throughput.he protocol is based on framed ALOHA and enables nodes to learn an optimal transmission strategy in a fully distributed manner without requiring detailed information about the external environment. The transmission strategy of sensor nodes is defined as a combination of time-slot and transmission-offset selection. By relying on the concept of learning through interaction with the environment, the proposed protocol enhances network resilience and adaptability. In both static and mobile network scenarios, it has been compared with the state-of-the-art framed ALOHA for the underwater environment (UW-ALOHA-Q), carrier-sensing ALOHA (CS-ALOHA), and delay-aware opportunistic transmission scheduling (DOTS) protocols. The simulation results show that the proposed solution leads to significant channel utilization gains, ranging from 13% to 106% in static network scenarios and from 23% to 126% in mobile network scenarios.oreover, using a more efficient learning strategy, it significantly reduces convergence time compared to UW-ALOHA-Q in larger networks, despite the increased action space.https://www.mdpi.com/1424-8220/23/9/4474medium access controlreinforcement learningunderwater acoustic networks |
| spellingShingle | Slavica Tomovic Igor Radusinovic DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks Sensors medium access control reinforcement learning underwater acoustic networks |
| title | DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks |
| title_full | DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks |
| title_fullStr | DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks |
| title_full_unstemmed | DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks |
| title_short | DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks |
| title_sort | dr aloha q a q learning based adaptive mac protocol for underwater acoustic sensor networks |
| topic | medium access control reinforcement learning underwater acoustic networks |
| url | https://www.mdpi.com/1424-8220/23/9/4474 |
| work_keys_str_mv | AT slavicatomovic dralohaqaqlearningbasedadaptivemacprotocolforunderwateracousticsensornetworks AT igorradusinovic dralohaqaqlearningbasedadaptivemacprotocolforunderwateracousticsensornetworks |