Decentralized Policy Coordination in Mobile Sensing with Consensual Communication
In a typical mobile-sensing scenario, multiple autonomous vehicles cooperatively navigate to maximize the spatial–temporal coverage of the environment. However, as each vehicle can only make decentralized navigation decisions based on limited local observations, it is still a critical challenge to c...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/24/9584 |
| _version_ | 1797455384669061120 |
|---|---|
| author | Bolei Zhang Lifa Wu Ilsun You |
| author_facet | Bolei Zhang Lifa Wu Ilsun You |
| author_sort | Bolei Zhang |
| collection | DOAJ |
| description | In a typical mobile-sensing scenario, multiple autonomous vehicles cooperatively navigate to maximize the spatial–temporal coverage of the environment. However, as each vehicle can only make decentralized navigation decisions based on limited local observations, it is still a critical challenge to coordinate the vehicles for cooperation in an open, dynamic environment. In this paper, we propose a novel framework that incorporates consensual communication in multi-agent reinforcement learning for cooperative mobile sensing. At each step, the vehicles first learn to communicate with each other, and then, based on the received messages from others, navigate. Through communication, the decentralized vehicles can share information to break through the dilemma of local observation. Moreover, we utilize mutual information as a regularizer to promote consensus among the vehicles. The mutual information can enforce positive correlation between the navigation policy and the communication message, and therefore implicitly coordinate the decentralized policies. The convergence of this regularized algorithm can be proved theoretically under certain mild assumptions. In the experiments, we show that our algorithm is scalable and can converge very fast during training phase. It also outperforms other baselines significantly in the execution phase. The results validate that consensual communication plays very important role in coordinating the behaviors of decentralized vehicles. |
| first_indexed | 2024-03-09T15:53:38Z |
| format | Article |
| id | doaj.art-82352f3d7c1540598ba1f5ee55cfe103 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T15:53:38Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-82352f3d7c1540598ba1f5ee55cfe1032023-11-24T17:51:58ZengMDPI AGSensors1424-82202022-12-012224958410.3390/s22249584Decentralized Policy Coordination in Mobile Sensing with Consensual CommunicationBolei Zhang0Lifa Wu1Ilsun You2School of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210023, ChinaSchool of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210023, ChinaDepartment of Financial Information Security, Kookmin University, Seoul 02707, Republic of KoreaIn a typical mobile-sensing scenario, multiple autonomous vehicles cooperatively navigate to maximize the spatial–temporal coverage of the environment. However, as each vehicle can only make decentralized navigation decisions based on limited local observations, it is still a critical challenge to coordinate the vehicles for cooperation in an open, dynamic environment. In this paper, we propose a novel framework that incorporates consensual communication in multi-agent reinforcement learning for cooperative mobile sensing. At each step, the vehicles first learn to communicate with each other, and then, based on the received messages from others, navigate. Through communication, the decentralized vehicles can share information to break through the dilemma of local observation. Moreover, we utilize mutual information as a regularizer to promote consensus among the vehicles. The mutual information can enforce positive correlation between the navigation policy and the communication message, and therefore implicitly coordinate the decentralized policies. The convergence of this regularized algorithm can be proved theoretically under certain mild assumptions. In the experiments, we show that our algorithm is scalable and can converge very fast during training phase. It also outperforms other baselines significantly in the execution phase. The results validate that consensual communication plays very important role in coordinating the behaviors of decentralized vehicles.https://www.mdpi.com/1424-8220/22/24/9584mobile sensingreinforcement learningdecentralized coordinationcommunication |
| spellingShingle | Bolei Zhang Lifa Wu Ilsun You Decentralized Policy Coordination in Mobile Sensing with Consensual Communication Sensors mobile sensing reinforcement learning decentralized coordination communication |
| title | Decentralized Policy Coordination in Mobile Sensing with Consensual Communication |
| title_full | Decentralized Policy Coordination in Mobile Sensing with Consensual Communication |
| title_fullStr | Decentralized Policy Coordination in Mobile Sensing with Consensual Communication |
| title_full_unstemmed | Decentralized Policy Coordination in Mobile Sensing with Consensual Communication |
| title_short | Decentralized Policy Coordination in Mobile Sensing with Consensual Communication |
| title_sort | decentralized policy coordination in mobile sensing with consensual communication |
| topic | mobile sensing reinforcement learning decentralized coordination communication |
| url | https://www.mdpi.com/1424-8220/22/24/9584 |
| work_keys_str_mv | AT boleizhang decentralizedpolicycoordinationinmobilesensingwithconsensualcommunication AT lifawu decentralizedpolicycoordinationinmobilesensingwithconsensualcommunication AT ilsunyou decentralizedpolicycoordinationinmobilesensingwithconsensualcommunication |