Fear-neuro-inspired reinforcement learning for safe autonomous driving
Ensuring safety and achieving human-level driving performance remain challenges for autonomous vehicles, especially in safety-critical situations. As a key component of artificial intelligence, reinforcement learning is promising and has shown great potential in many complex tasks; however, its lack...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173312 |
| _version_ | 1811692273356767232 |
|---|---|
| author | He, Xiangkun Wu, Jingda Huang, Zhiyu Hu, Zhongxu Wang, Jun Sangiovanni-Vincentelli, Alberto Lv, Chen |
| author2 | School of Mechanical and Aerospace Engineering |
| author_facet | School of Mechanical and Aerospace Engineering He, Xiangkun Wu, Jingda Huang, Zhiyu Hu, Zhongxu Wang, Jun Sangiovanni-Vincentelli, Alberto Lv, Chen |
| author_sort | He, Xiangkun |
| collection | NTU |
| description | Ensuring safety and achieving human-level driving performance remain challenges for autonomous vehicles, especially in safety-critical situations. As a key component of artificial intelligence, reinforcement learning is promising and has shown great potential in many complex tasks; however, its lack of safety guarantees limits its real-world applicability. Hence, further advancing reinforcement learning, especially from the safety perspective, is of great importance for autonomous driving. As revealed by cognitive neuroscientists, the amygdala of the brain can elicit defensive responses against threats or hazards, which is crucial for survival in and adaptation to risky environments. Drawing inspiration from this scientific discovery, we present a fear-neuro-inspired reinforcement learning framework to realize safe autonomous driving through modeling the amygdala functionality. This new technique facilitates an agent to learn defensive behaviors and achieve safe decision making with fewer safety violations. Through experimental tests, we show that the proposed approach enables the autonomous driving agent to attain state-of-the-art performance compared to the baseline agents and perform comparably to 30 certified human drivers, across various safety-critical scenarios. The results demonstrate the feasibility and effectiveness of our framework while also shedding light on the crucial role of simulating the amygdala function in the application of reinforcement learning to safety-critical autonomous driving domains. |
| first_indexed | 2024-10-01T06:33:10Z |
| format | Journal Article |
| id | ntu-10356/173312 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:33:10Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1733122024-01-24T00:14:55Z Fear-neuro-inspired reinforcement learning for safe autonomous driving He, Xiangkun Wu, Jingda Huang, Zhiyu Hu, Zhongxu Wang, Jun Sangiovanni-Vincentelli, Alberto Lv, Chen School of Mechanical and Aerospace Engineering Engineering::Mechanical engineering Trustworthy Artificial Intelligence Reinforcement Learning Ensuring safety and achieving human-level driving performance remain challenges for autonomous vehicles, especially in safety-critical situations. As a key component of artificial intelligence, reinforcement learning is promising and has shown great potential in many complex tasks; however, its lack of safety guarantees limits its real-world applicability. Hence, further advancing reinforcement learning, especially from the safety perspective, is of great importance for autonomous driving. As revealed by cognitive neuroscientists, the amygdala of the brain can elicit defensive responses against threats or hazards, which is crucial for survival in and adaptation to risky environments. Drawing inspiration from this scientific discovery, we present a fear-neuro-inspired reinforcement learning framework to realize safe autonomous driving through modeling the amygdala functionality. This new technique facilitates an agent to learn defensive behaviors and achieve safe decision making with fewer safety violations. Through experimental tests, we show that the proposed approach enables the autonomous driving agent to attain state-of-the-art performance compared to the baseline agents and perform comparably to 30 certified human drivers, across various safety-critical scenarios. The results demonstrate the feasibility and effectiveness of our framework while also shedding light on the crucial role of simulating the amygdala function in the application of reinforcement learning to safety-critical autonomous driving domains. Agency for Science, Technology and Research (A*STAR) Ministry of Education (MOE) Nanyang Technological University National Research Foundation (NRF) This work was supported in part by the Start-Up Grant-Nanyang Assistant Professorship Grant of Nanyang Technological University, the Agency for Science, Technology and Research (A*STAR), Singapore, under Advanced Manufacturing and Engineering (AME) Young Individual Research under Grant A2084c0156, the MTC Individual Research under Grant M22K2c0079, the ANR-NRF Joint under Grant NRF2021-NRF-ANR003 HM Science, and the Ministry of Education (MOE), Singapore, under the Tier 2 under Grant MOET2EP50222-0002. 2024-01-24T00:14:55Z 2024-01-24T00:14:55Z 2024 Journal Article He, X., Wu, J., Huang, Z., Hu, Z., Wang, J., Sangiovanni-Vincentelli, A. & Lv, C. (2024). Fear-neuro-inspired reinforcement learning for safe autonomous driving. IEEE Transactions On Pattern Analysis and Machine Intelligence, 46(1), 267-279. https://dx.doi.org/10.1109/TPAMI.2023.3322426 0162-8828 https://hdl.handle.net/10356/173312 10.1109/TPAMI.2023.3322426 37801378 2-s2.0-85174819545 1 46 267 279 en A2084c0156 M22K2c0079 NRF2021-NRF-ANR003 MOET2EP50222-0002 IEEE Transactions on Pattern Analysis and Machine Intelligence © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Mechanical engineering Trustworthy Artificial Intelligence Reinforcement Learning He, Xiangkun Wu, Jingda Huang, Zhiyu Hu, Zhongxu Wang, Jun Sangiovanni-Vincentelli, Alberto Lv, Chen Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title | Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title_full | Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title_fullStr | Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title_full_unstemmed | Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title_short | Fear-neuro-inspired reinforcement learning for safe autonomous driving |
| title_sort | fear neuro inspired reinforcement learning for safe autonomous driving |
| topic | Engineering::Mechanical engineering Trustworthy Artificial Intelligence Reinforcement Learning |
| url | https://hdl.handle.net/10356/173312 |
| work_keys_str_mv | AT hexiangkun fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT wujingda fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT huangzhiyu fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT huzhongxu fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT wangjun fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT sangiovannivincentellialberto fearneuroinspiredreinforcementlearningforsafeautonomousdriving AT lvchen fearneuroinspiredreinforcementlearningforsafeautonomousdriving |