Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic
Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that foll...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Open Journal of Intelligent Transportation Systems |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9770186/ |
| _version_ | 1797974396217851904 |
|---|---|
| author | Rodolfo Valiente Behrad Toghi Ramtin Pedarsani Yaser P. Fallah |
| author_facet | Rodolfo Valiente Behrad Toghi Ramtin Pedarsani Yaser P. Fallah |
| author_sort | Rodolfo Valiente |
| collection | DOAJ |
| description | Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations. |
| first_indexed | 2024-04-11T04:19:21Z |
| format | Article |
| id | doaj.art-4bf0e8f9b31a4ec6bbf2efef1fa5a5da |
| institution | Directory Open Access Journal |
| issn | 2687-7813 |
| language | English |
| last_indexed | 2024-04-11T04:19:21Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Intelligent Transportation Systems |
| spelling | doaj.art-4bf0e8f9b31a4ec6bbf2efef1fa5a5da2022-12-31T00:02:03ZengIEEEIEEE Open Journal of Intelligent Transportation Systems2687-78132022-01-01339741010.1109/OJITS.2022.31729819770186Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy TrafficRodolfo Valiente0https://orcid.org/0000-0003-0617-037XBehrad Toghi1Ramtin Pedarsani2https://orcid.org/0000-0002-1126-0292Yaser P. Fallah3Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL, USADepartment of Electrical and Computer Engineering, University of Central Florida, Orlando, FL, USADepartment of Electrical and Computer Engineering, University of California at Santa Barbara, Santa Barbara, CA, USADepartment of Electrical and Computer Engineering, University of Central Florida, Orlando, FL, USABuilding autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.https://ieeexplore.ieee.org/document/9770186/Behavior planningcooperative drivingmixed-autonomyreinforcement learningrobustness |
| spellingShingle | Rodolfo Valiente Behrad Toghi Ramtin Pedarsani Yaser P. Fallah Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic IEEE Open Journal of Intelligent Transportation Systems Behavior planning cooperative driving mixed-autonomy reinforcement learning robustness |
| title | Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic |
| title_full | Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic |
| title_fullStr | Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic |
| title_full_unstemmed | Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic |
| title_short | Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic |
| title_sort | robustness and adaptability of reinforcement learning based cooperative autonomous driving in mixed autonomy traffic |
| topic | Behavior planning cooperative driving mixed-autonomy reinforcement learning robustness |
| url | https://ieeexplore.ieee.org/document/9770186/ |
| work_keys_str_mv | AT rodolfovaliente robustnessandadaptabilityofreinforcementlearningbasedcooperativeautonomousdrivinginmixedautonomytraffic AT behradtoghi robustnessandadaptabilityofreinforcementlearningbasedcooperativeautonomousdrivinginmixedautonomytraffic AT ramtinpedarsani robustnessandadaptabilityofreinforcementlearningbasedcooperativeautonomousdrivinginmixedautonomytraffic AT yaserpfallah robustnessandadaptabilityofreinforcementlearningbasedcooperativeautonomousdrivinginmixedautonomytraffic |