Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm
This paper presents and assesses the feasibility and potential of a novel concept: the operation of multiple Unmanned Aerial Vehicles (UAVs) commanded and supported by a manned “Tender” air vehicle carrying a pilot and flight manager(s). The “Tender” is equipped to flexibly and economically monitor...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Canadian Science Publishing
2023-01-01
|
| Series: | Drone Systems and Applications |
| Subjects: | |
| Online Access: | https://cdnsciencepub.com/doi/10.1139/dsa-2022-0043 |
| _version_ | 1797871889292460032 |
|---|---|
| author | Richard C. Millar Leila Hashemi Armin Mahmoodi Robert Walter Meyer Jeremy Laliberte |
| author_facet | Richard C. Millar Leila Hashemi Armin Mahmoodi Robert Walter Meyer Jeremy Laliberte |
| author_sort | Richard C. Millar |
| collection | DOAJ |
| description | This paper presents and assesses the feasibility and potential of a novel concept: the operation of multiple Unmanned Aerial Vehicles (UAVs) commanded and supported by a manned “Tender” air vehicle carrying a pilot and flight manager(s). The “Tender” is equipped to flexibly and economically monitor and manage multiple diverse UAVs over otherwise inaccessible terrain through wireless communication. The proposed architecture enables operations and analysis supported by the means to detect, assess, and accommodate change and hazards on the spot with effective human observation and coordination. Further, this paper seeks to find the optimal trajectories for UAVs to collect data from sensors in a predefined continuous space. We formulate the path-planning problem for a cooperative, and a diverse swarm of UAVs tasked with optimizing multiple objectives simultaneously with the goal of maximizing accumulated data within a given flight time within cloud data processing constraints as well as minimizing the probable imposed risk during UAVs mission. The risk assessment model determines risk indicators using an integrated Specific Operation Risk Assessment—Bayesian belief network approach, while its resultant analysis is weighted through the analytic hierarchy process ranking model. To this end, as the problem is formulated as a convex optimization model, and we propose a low complexity multi-objective reinforcement learning (MORL) algorithm with a provable performance guarantee to solve the problem efficiently. We show that the MORL architecture can be successfully trained and allows each UAV to map each observation of the network state to an action to make optimal movement decisions. This proposed network architecture enables the UAVs to balance multiple objectives. Estimated MSE measures show that the algorithm produced decreasing errors in the learning process with increasing epoch number. |
| first_indexed | 2024-04-10T00:50:08Z |
| format | Article |
| id | doaj.art-d234accafdec4ceeb3527790f8532367 |
| institution | Directory Open Access Journal |
| issn | 2564-4939 |
| language | English |
| last_indexed | 2024-04-10T00:50:08Z |
| publishDate | 2023-01-01 |
| publisher | Canadian Science Publishing |
| record_format | Article |
| series | Drone Systems and Applications |
| spelling | doaj.art-d234accafdec4ceeb3527790f85323672023-03-13T11:29:45ZengCanadian Science PublishingDrone Systems and Applications2564-49392023-01-011111710.1139/dsa-2022-0043Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithmRichard C. Millar0Leila Hashemi1Armin Mahmoodi2Robert Walter Meyer3Jeremy Laliberte4The George Washington University, Engineering Management & Systems Engineering, WA, DC, USACarleton University, Mechanical & Aerospace Engineering, Ottawa, ON, CanadaCarleton University, Mechanical & Aerospace Engineering, Ottawa, ON, CanadaState University of New York, Chemical Engineering, Syracuse, NY, USACarleton University, Mechanical & Aerospace Engineering, Ottawa, ON, CanadaThis paper presents and assesses the feasibility and potential of a novel concept: the operation of multiple Unmanned Aerial Vehicles (UAVs) commanded and supported by a manned “Tender” air vehicle carrying a pilot and flight manager(s). The “Tender” is equipped to flexibly and economically monitor and manage multiple diverse UAVs over otherwise inaccessible terrain through wireless communication. The proposed architecture enables operations and analysis supported by the means to detect, assess, and accommodate change and hazards on the spot with effective human observation and coordination. Further, this paper seeks to find the optimal trajectories for UAVs to collect data from sensors in a predefined continuous space. We formulate the path-planning problem for a cooperative, and a diverse swarm of UAVs tasked with optimizing multiple objectives simultaneously with the goal of maximizing accumulated data within a given flight time within cloud data processing constraints as well as minimizing the probable imposed risk during UAVs mission. The risk assessment model determines risk indicators using an integrated Specific Operation Risk Assessment—Bayesian belief network approach, while its resultant analysis is weighted through the analytic hierarchy process ranking model. To this end, as the problem is formulated as a convex optimization model, and we propose a low complexity multi-objective reinforcement learning (MORL) algorithm with a provable performance guarantee to solve the problem efficiently. We show that the MORL architecture can be successfully trained and allows each UAV to map each observation of the network state to an action to make optimal movement decisions. This proposed network architecture enables the UAVs to balance multiple objectives. Estimated MSE measures show that the algorithm produced decreasing errors in the learning process with increasing epoch number.https://cdnsciencepub.com/doi/10.1139/dsa-2022-0043trajectory optimizationmulti-objective reinforcement algorithmBayesian belief networkunmanned aerial vehicle (UAV)LIDAR sensor |
| spellingShingle | Richard C. Millar Leila Hashemi Armin Mahmoodi Robert Walter Meyer Jeremy Laliberte Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm Drone Systems and Applications trajectory optimization multi-objective reinforcement algorithm Bayesian belief network unmanned aerial vehicle (UAV) LIDAR sensor |
| title | Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm |
| title_full | Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm |
| title_fullStr | Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm |
| title_full_unstemmed | Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm |
| title_short | Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm |
| title_sort | integrating unmanned and manned uavs data network based on combined bayesian belief network and multi objective reinforcement learning algorithm |
| topic | trajectory optimization multi-objective reinforcement algorithm Bayesian belief network unmanned aerial vehicle (UAV) LIDAR sensor |
| url | https://cdnsciencepub.com/doi/10.1139/dsa-2022-0043 |
| work_keys_str_mv | AT richardcmillar integratingunmannedandmanneduavsdatanetworkbasedoncombinedbayesianbeliefnetworkandmultiobjectivereinforcementlearningalgorithm AT leilahashemi integratingunmannedandmanneduavsdatanetworkbasedoncombinedbayesianbeliefnetworkandmultiobjectivereinforcementlearningalgorithm AT arminmahmoodi integratingunmannedandmanneduavsdatanetworkbasedoncombinedbayesianbeliefnetworkandmultiobjectivereinforcementlearningalgorithm AT robertwaltermeyer integratingunmannedandmanneduavsdatanetworkbasedoncombinedbayesianbeliefnetworkandmultiobjectivereinforcementlearningalgorithm AT jeremylaliberte integratingunmannedandmanneduavsdatanetworkbasedoncombinedbayesianbeliefnetworkandmultiobjectivereinforcementlearningalgorithm |