Towards safe autonomous driving: decision making with observation-robust reinforcement learning
Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving. To address these issues and further improve safety, automated driving is required to be capable of handling perception uncertainties....
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173164 |
| _version_ | 1826111109862522880 |
|---|---|
| author | He, Xiangkun Lv, Chen |
| author2 | School of Mechanical and Aerospace Engineering |
| author_facet | School of Mechanical and Aerospace Engineering He, Xiangkun Lv, Chen |
| author_sort | He, Xiangkun |
| collection | NTU |
| description | Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving. To address these issues and further improve safety, automated driving is required to be capable of handling perception uncertainties. Here, this paper presents an observation-robust reinforcement learning against observational uncertainties to realize safe decision making for autonomous vehicles. Specifically, an adversarial agent is trained online to generate optimal adversarial attacks on observations, which attempts to amplify the average variation distance on perturbed policies. In addition, an observation-robust actor-critic approach is developed to enable the agent to learn the optimal policies and ensure that the changes of the policies perturbed by optimal adversarial attacks remain within a certain bound. Lastly, the safe decision making scheme is evaluated on a lane change task under complex highway traffic scenarios. The results show that the developed approach can ensure autonomous driving performance, as well as the policy robustness against adversarial attacks on observations. |
| first_indexed | 2024-10-01T02:45:15Z |
| format | Journal Article |
| id | ntu-10356/173164 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:45:15Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1731642024-01-20T16:48:04Z Towards safe autonomous driving: decision making with observation-robust reinforcement learning He, Xiangkun Lv, Chen School of Mechanical and Aerospace Engineering Engineering::Mechanical engineering Autonomous Vehicle Safe Decision Making Most real-world situations involve unavoidable measurement noises or perception errors which result in unsafe decision making or even casualty in autonomous driving. To address these issues and further improve safety, automated driving is required to be capable of handling perception uncertainties. Here, this paper presents an observation-robust reinforcement learning against observational uncertainties to realize safe decision making for autonomous vehicles. Specifically, an adversarial agent is trained online to generate optimal adversarial attacks on observations, which attempts to amplify the average variation distance on perturbed policies. In addition, an observation-robust actor-critic approach is developed to enable the agent to learn the optimal policies and ensure that the changes of the policies perturbed by optimal adversarial attacks remain within a certain bound. Lastly, the safe decision making scheme is evaluated on a lane change task under complex highway traffic scenarios. The results show that the developed approach can ensure autonomous driving performance, as well as the policy robustness against adversarial attacks on observations. Published version This work was supported by Foundation of State Key Laboratory of Automotive Simulation and Control. 2024-01-16T01:41:02Z 2024-01-16T01:41:02Z 2023 Journal Article He, X. & Lv, C. (2023). Towards safe autonomous driving: decision making with observation-robust reinforcement learning. Automotive Innovation, 6(4), 509-520. https://dx.doi.org/10.1007/s42154-023-00256-x 2096-4250 https://hdl.handle.net/10356/173164 10.1007/s42154-023-00256-x 2-s2.0-85176125985 4 6 509 520 en Automotive Innovation © The Author(s) 2023. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. application/pdf |
| spellingShingle | Engineering::Mechanical engineering Autonomous Vehicle Safe Decision Making He, Xiangkun Lv, Chen Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title | Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title_full | Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title_fullStr | Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title_full_unstemmed | Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title_short | Towards safe autonomous driving: decision making with observation-robust reinforcement learning |
| title_sort | towards safe autonomous driving decision making with observation robust reinforcement learning |
| topic | Engineering::Mechanical engineering Autonomous Vehicle Safe Decision Making |
| url | https://hdl.handle.net/10356/173164 |
| work_keys_str_mv | AT hexiangkun towardssafeautonomousdrivingdecisionmakingwithobservationrobustreinforcementlearning AT lvchen towardssafeautonomousdrivingdecisionmakingwithobservationrobustreinforcementlearning |