Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery
The popularity of commercial unmanned aerial vehicles has drawn great attention from the e-commerce industry due to their suitability for last-mile delivery. However, the organization of multiple aerial vehicles efficiently for delivery within limitations and uncertainties is still a problem. The ma...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Drones |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-446X/8/3/71 |
| _version_ | 1797241423869771776 |
|---|---|
| author | Farabi Ahmed Tarhan Nazım Kemal Ure |
| author_facet | Farabi Ahmed Tarhan Nazım Kemal Ure |
| author_sort | Farabi Ahmed Tarhan |
| collection | DOAJ |
| description | The popularity of commercial unmanned aerial vehicles has drawn great attention from the e-commerce industry due to their suitability for last-mile delivery. However, the organization of multiple aerial vehicles efficiently for delivery within limitations and uncertainties is still a problem. The main challenge of planning is scalability, since the planning space grows exponentially to the number of agents, and it is not efficient to let human-level supervisors structure the problem for large-scale settings. Algorithms based on Deep Q-Networks had unprecedented success in solving decision-making problems. Extension of these algorithms to multi-agent problems is limited due to scalability issues. This work proposes an approach that improves the performance of Deep Q-Networks on multi-agent delivery by drone problems by utilizing state decompositions for lowering the problem complexity, Curriculum Learning for handling the exploration complexity, and Genetic Algorithms for searching efficient packet-drone matching across the combinatorial solution space. The performance of the proposed method is shown in a multi-agent delivery by drone problem that has 10 agents and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>≈</mo><msup><mn>10</mn><mn>77</mn></msup></mrow></semantics></math></inline-formula> state–action pairs. Comparative simulation results are provided to demonstrate the merit of the proposed method. The proposed Genetic-Algorithm-aided multi-agent DRL outperformed the rest in terms of scalability and convergent behavior. |
| first_indexed | 2024-04-24T18:23:06Z |
| format | Article |
| id | doaj.art-0bd64ca60a9848a991917d283fb528d3 |
| institution | Directory Open Access Journal |
| issn | 2504-446X |
| language | English |
| last_indexed | 2024-04-24T18:23:06Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Drones |
| spelling | doaj.art-0bd64ca60a9848a991917d283fb528d32024-03-27T13:33:54ZengMDPI AGDrones2504-446X2024-02-01837110.3390/drones8030071Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone DeliveryFarabi Ahmed Tarhan0Nazım Kemal Ure1Department of Aeronautics Engineering, Istanbul Technical University, ITU Ayazaga Campus, Istanbul 34469, TurkeyArtificial Intelligence and Data Science Application and Research Center, Istanbul Technical University, ITU Ayazaga Campus, Istanbul 34469, TurkeyThe popularity of commercial unmanned aerial vehicles has drawn great attention from the e-commerce industry due to their suitability for last-mile delivery. However, the organization of multiple aerial vehicles efficiently for delivery within limitations and uncertainties is still a problem. The main challenge of planning is scalability, since the planning space grows exponentially to the number of agents, and it is not efficient to let human-level supervisors structure the problem for large-scale settings. Algorithms based on Deep Q-Networks had unprecedented success in solving decision-making problems. Extension of these algorithms to multi-agent problems is limited due to scalability issues. This work proposes an approach that improves the performance of Deep Q-Networks on multi-agent delivery by drone problems by utilizing state decompositions for lowering the problem complexity, Curriculum Learning for handling the exploration complexity, and Genetic Algorithms for searching efficient packet-drone matching across the combinatorial solution space. The performance of the proposed method is shown in a multi-agent delivery by drone problem that has 10 agents and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>≈</mo><msup><mn>10</mn><mn>77</mn></msup></mrow></semantics></math></inline-formula> state–action pairs. Comparative simulation results are provided to demonstrate the merit of the proposed method. The proposed Genetic-Algorithm-aided multi-agent DRL outperformed the rest in terms of scalability and convergent behavior.https://www.mdpi.com/2504-446X/8/3/71reinforcement learninggenetic algorithmsdeep q-networksdelivery by dronemulti-agent decision making |
| spellingShingle | Farabi Ahmed Tarhan Nazım Kemal Ure Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery Drones reinforcement learning genetic algorithms deep q-networks delivery by drone multi-agent decision making |
| title | Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery |
| title_full | Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery |
| title_fullStr | Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery |
| title_full_unstemmed | Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery |
| title_short | Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery |
| title_sort | genetic algorithm aided deep reinforcement learning for multi agent drone delivery |
| topic | reinforcement learning genetic algorithms deep q-networks delivery by drone multi-agent decision making |
| url | https://www.mdpi.com/2504-446X/8/3/71 |
| work_keys_str_mv | AT farabiahmedtarhan geneticalgorithmaideddeepreinforcementlearningformultiagentdronedelivery AT nazımkemalure geneticalgorithmaideddeepreinforcementlearningformultiagentdronedelivery |