A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach
Considerable attention has been given to leverage a variety of smart city applications using unmanned aerial vehicles (UAVs). The rapid advances in artificial intelligence can empower UAVs with autonomous capabilities allowing them to learn from their surrounding environment and act accordingly with...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Open Journal of Vehicular Technology |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9028197/ |
| _version_ | 1818457963834114048 |
|---|---|
| author | Omar Bouhamed Hakim Ghazzai Hichem Besbes Yehia Massoud |
| author_facet | Omar Bouhamed Hakim Ghazzai Hichem Besbes Yehia Massoud |
| author_sort | Omar Bouhamed |
| collection | DOAJ |
| description | Considerable attention has been given to leverage a variety of smart city applications using unmanned aerial vehicles (UAVs). The rapid advances in artificial intelligence can empower UAVs with autonomous capabilities allowing them to learn from their surrounding environment and act accordingly without human intervention. In this paper, we propose a spatiotemporal scheduling framework for autonomous UAVs using reinforcement learning. The framework enables UAVs to autonomously determine their schedules to cover the maximum of pre-scheduled events spatially and temporally distributed in a given geographical area and over a pre-determined time horizon. The designed framework has the ability to update the planned schedules in case of unexpected emergency events. The UAVs are trained using the Q-learning (QL) algorithm to find effective scheduling plan. A customized reward function is developed to consider several constraints especially the limited battery capacity of the flying units, the time windows of events, and the delays caused by the UAV navigation between events. Numerical simulations show the behavior of the autonomous UAVs for various scenarios and corroborate the ability of QL to handle complex vehicle routing problems with several constraints. A comparison with an optimal deterministic solution is also provided to validate the performance of the learning-based solution. |
| first_indexed | 2024-12-14T22:50:55Z |
| format | Article |
| id | doaj.art-36f6fe12d39e42f89ee0742b0d2861e9 |
| institution | Directory Open Access Journal |
| issn | 2644-1330 |
| language | English |
| last_indexed | 2024-12-14T22:50:55Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Vehicular Technology |
| spelling | doaj.art-36f6fe12d39e42f89ee0742b0d2861e92022-12-21T22:44:43ZengIEEEIEEE Open Journal of Vehicular Technology2644-13302020-01-0119310610.1109/OJVT.2020.29795599028197A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based ApproachOmar Bouhamed0https://orcid.org/0000-0002-6595-5663Hakim Ghazzai1https://orcid.org/0000-0002-8636-4264Hichem Besbes2https://orcid.org/0000-0003-1063-494XYehia Massoud3School of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ, USASchool of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ, USAHigher School of Communication of Tunis, University of Carthage, Tunis, TunisiaSchool of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ, USAConsiderable attention has been given to leverage a variety of smart city applications using unmanned aerial vehicles (UAVs). The rapid advances in artificial intelligence can empower UAVs with autonomous capabilities allowing them to learn from their surrounding environment and act accordingly without human intervention. In this paper, we propose a spatiotemporal scheduling framework for autonomous UAVs using reinforcement learning. The framework enables UAVs to autonomously determine their schedules to cover the maximum of pre-scheduled events spatially and temporally distributed in a given geographical area and over a pre-determined time horizon. The designed framework has the ability to update the planned schedules in case of unexpected emergency events. The UAVs are trained using the Q-learning (QL) algorithm to find effective scheduling plan. A customized reward function is developed to consider several constraints especially the limited battery capacity of the flying units, the time windows of events, and the delays caused by the UAV navigation between events. Numerical simulations show the behavior of the autonomous UAVs for various scenarios and corroborate the ability of QL to handle complex vehicle routing problems with several constraints. A comparison with an optimal deterministic solution is also provided to validate the performance of the learning-based solution.https://ieeexplore.ieee.org/document/9028197/Reinforcement learningscheduling solutionsmart cityunmanned aerial vehicles (UAVs)vehicle routing problem |
| spellingShingle | Omar Bouhamed Hakim Ghazzai Hichem Besbes Yehia Massoud A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach IEEE Open Journal of Vehicular Technology Reinforcement learning scheduling solution smart city unmanned aerial vehicles (UAVs) vehicle routing problem |
| title | A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach |
| title_full | A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach |
| title_fullStr | A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach |
| title_full_unstemmed | A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach |
| title_short | A Generic Spatiotemporal Scheduling for Autonomous UAVs: A Reinforcement Learning-Based Approach |
| title_sort | generic spatiotemporal scheduling for autonomous uavs a reinforcement learning based approach |
| topic | Reinforcement learning scheduling solution smart city unmanned aerial vehicles (UAVs) vehicle routing problem |
| url | https://ieeexplore.ieee.org/document/9028197/ |
| work_keys_str_mv | AT omarbouhamed agenericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT hakimghazzai agenericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT hichembesbes agenericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT yehiamassoud agenericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT omarbouhamed genericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT hakimghazzai genericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT hichembesbes genericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach AT yehiamassoud genericspatiotemporalschedulingforautonomousuavsareinforcementlearningbasedapproach |