Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning
The cooperative active defense guidance problem for a spacecraft with active defense is investigated in this paper. An engagement between a spacecraft, an active defense vehicle, and an interceptor is considered, where the target spacecraft with active defense will attempt to evade the interceptor....
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/19/4211 |
| _version_ | 1797575566711324672 |
|---|---|
| author | Weilin Ni Jiaqi Liu Zhi Li Peng Liu Haizhao Liang |
| author_facet | Weilin Ni Jiaqi Liu Zhi Li Peng Liu Haizhao Liang |
| author_sort | Weilin Ni |
| collection | DOAJ |
| description | The cooperative active defense guidance problem for a spacecraft with active defense is investigated in this paper. An engagement between a spacecraft, an active defense vehicle, and an interceptor is considered, where the target spacecraft with active defense will attempt to evade the interceptor. Prior knowledge uncertainty and observation noise are taken into account simultaneously, which are vital for traditional guidance strategies such as the differential-game-based guidance method. In this set, we propose an intelligent cooperative active defense (ICAAI) guidance strategy based on deep reinforcement learning. ICAAI effectively coordinates defender and target maneuvers to achieve successful evasion with less prior knowledge and observational noise. Furthermore, we introduce an efficient and stable convergence (ESC) training approach employing reward shaping and curriculum learning to tackle the sparse reward problem in ICAAI training. Numerical experiments are included to demonstrate ICAAI’s real-time performance, convergence, adaptiveness, and robustness through the learning process and Monte Carlo simulations. The learning process showcases improved convergence efficiency with ESC, while simulation results illustrate ICAAI’s enhanced robustness and adaptiveness compared to optimal guidance laws. |
| first_indexed | 2024-03-10T21:40:20Z |
| format | Article |
| id | doaj.art-fcd8882cf8c44850ac0a6f4b4228c099 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T21:40:20Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-fcd8882cf8c44850ac0a6f4b4228c0992023-11-19T14:44:53ZengMDPI AGMathematics2227-73902023-10-011119421110.3390/math11194211Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement LearningWeilin Ni0Jiaqi Liu1Zhi Li2Peng Liu3Haizhao Liang4School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzheng 518107, ChinaNational Key Laboratory of Science and Technology on Test Physics and Numerical Mathematics, Beijing 100076, ChinaSchool of Aeronautics and Astronautics, Sun Yat-sen University, Shenzheng 518107, ChinaNational Key Laboratory of Science and Technology on Test Physics and Numerical Mathematics, Beijing 100076, ChinaSchool of Aeronautics and Astronautics, Sun Yat-sen University, Shenzheng 518107, ChinaThe cooperative active defense guidance problem for a spacecraft with active defense is investigated in this paper. An engagement between a spacecraft, an active defense vehicle, and an interceptor is considered, where the target spacecraft with active defense will attempt to evade the interceptor. Prior knowledge uncertainty and observation noise are taken into account simultaneously, which are vital for traditional guidance strategies such as the differential-game-based guidance method. In this set, we propose an intelligent cooperative active defense (ICAAI) guidance strategy based on deep reinforcement learning. ICAAI effectively coordinates defender and target maneuvers to achieve successful evasion with less prior knowledge and observational noise. Furthermore, we introduce an efficient and stable convergence (ESC) training approach employing reward shaping and curriculum learning to tackle the sparse reward problem in ICAAI training. Numerical experiments are included to demonstrate ICAAI’s real-time performance, convergence, adaptiveness, and robustness through the learning process and Monte Carlo simulations. The learning process showcases improved convergence efficiency with ESC, while simulation results illustrate ICAAI’s enhanced robustness and adaptiveness compared to optimal guidance laws.https://www.mdpi.com/2227-7390/11/19/4211cooperative guidancereinforcement learningactive protectionguidance law |
| spellingShingle | Weilin Ni Jiaqi Liu Zhi Li Peng Liu Haizhao Liang Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning Mathematics cooperative guidance reinforcement learning active protection guidance law |
| title | Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning |
| title_full | Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning |
| title_fullStr | Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning |
| title_full_unstemmed | Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning |
| title_short | Cooperative Guidance Strategy for Active Spacecraft Protection from a Homing Interceptor via Deep Reinforcement Learning |
| title_sort | cooperative guidance strategy for active spacecraft protection from a homing interceptor via deep reinforcement learning |
| topic | cooperative guidance reinforcement learning active protection guidance law |
| url | https://www.mdpi.com/2227-7390/11/19/4211 |
| work_keys_str_mv | AT weilinni cooperativeguidancestrategyforactivespacecraftprotectionfromahominginterceptorviadeepreinforcementlearning AT jiaqiliu cooperativeguidancestrategyforactivespacecraftprotectionfromahominginterceptorviadeepreinforcementlearning AT zhili cooperativeguidancestrategyforactivespacecraftprotectionfromahominginterceptorviadeepreinforcementlearning AT pengliu cooperativeguidancestrategyforactivespacecraftprotectionfromahominginterceptorviadeepreinforcementlearning AT haizhaoliang cooperativeguidancestrategyforactivespacecraftprotectionfromahominginterceptorviadeepreinforcementlearning |