Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles
Multiple vehicle collision avoidance strategies with safe lane changing strategy for vehicle control using learning base technique are the most crucial concern in autonomous driving system. Statistics shows that the latest autonomous driving systems are usually prone to rear-end collision. Rear-end...
| Main Authors: | , , |
|---|---|
| Format: | Conference or Workshop Item |
| Language: | English |
| Published: |
IEEE
2021
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/33324/1/Comparison_of_PPO_and_SAC_Algorithms_Towards_Decision_Making_Strategies_for_Collision_Avoidance_Among_Multiple_Autonomous_Vehicles%20%281%29.pdf |
| _version_ | 1825814187934220288 |
|---|---|
| author | Abu Jafar, Md Muzahid Syafiq Fauzi, Kamarulzaman Md Arafatur, Rahman |
| author_facet | Abu Jafar, Md Muzahid Syafiq Fauzi, Kamarulzaman Md Arafatur, Rahman |
| author_sort | Abu Jafar, Md Muzahid |
| collection | UMP |
| description | Multiple vehicle collision avoidance strategies with safe lane changing strategy for vehicle control using learning base technique are the most crucial concern in autonomous driving system. Statistics shows that the latest autonomous driving systems are usually prone to rear-end collision. Rear-end collisions often result in severe injuries as well as traffic jam and the consequences are much worse for multiple-vehicle collision. Many previous autonomous driving research focused solely on collision avoidance strategies for two consecutive vehicles. This study proposes a centralised control strategy for multiple vehicles using reinforcement learning focused on partner consideration and goal attainment. The system depicted as a group of vehicles are communicate and coordinate each others by a set of rays and maintain a short following move away. In order to address this challenge, a simulation was implemented in the Unity3D game engine and two state-of-the-art RL algorithms PPO (Proximal Policy Optimization) and SAC (Soft Actor-Critic) were trained by an agent using Unity ML-Agents Toolkit. In terms of success rate, performance, training speed and stability two algorithms are comparable. The potency of algorithms has been assessed by the traffic flow (1) change in vehicle speed, (2) differ in the vehicle beginning positions, and (3) switch to next lane. The agent performed similarly at a 91% success rate in PPO or SAC applications |
| first_indexed | 2024-03-06T12:55:08Z |
| format | Conference or Workshop Item |
| id | UMPir33324 |
| institution | Universiti Malaysia Pahang |
| language | English |
| last_indexed | 2024-03-06T12:55:08Z |
| publishDate | 2021 |
| publisher | IEEE |
| record_format | dspace |
| spelling | UMPir333242022-02-08T04:04:36Z http://umpir.ump.edu.my/id/eprint/33324/ Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles Abu Jafar, Md Muzahid Syafiq Fauzi, Kamarulzaman Md Arafatur, Rahman TJ Mechanical engineering and machinery Multiple vehicle collision avoidance strategies with safe lane changing strategy for vehicle control using learning base technique are the most crucial concern in autonomous driving system. Statistics shows that the latest autonomous driving systems are usually prone to rear-end collision. Rear-end collisions often result in severe injuries as well as traffic jam and the consequences are much worse for multiple-vehicle collision. Many previous autonomous driving research focused solely on collision avoidance strategies for two consecutive vehicles. This study proposes a centralised control strategy for multiple vehicles using reinforcement learning focused on partner consideration and goal attainment. The system depicted as a group of vehicles are communicate and coordinate each others by a set of rays and maintain a short following move away. In order to address this challenge, a simulation was implemented in the Unity3D game engine and two state-of-the-art RL algorithms PPO (Proximal Policy Optimization) and SAC (Soft Actor-Critic) were trained by an agent using Unity ML-Agents Toolkit. In terms of success rate, performance, training speed and stability two algorithms are comparable. The potency of algorithms has been assessed by the traffic flow (1) change in vehicle speed, (2) differ in the vehicle beginning positions, and (3) switch to next lane. The agent performed similarly at a 91% success rate in PPO or SAC applications IEEE 2021 Conference or Workshop Item PeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/33324/1/Comparison_of_PPO_and_SAC_Algorithms_Towards_Decision_Making_Strategies_for_Collision_Avoidance_Among_Multiple_Autonomous_Vehicles%20%281%29.pdf Abu Jafar, Md Muzahid and Syafiq Fauzi, Kamarulzaman and Md Arafatur, Rahman (2021) Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles. In: 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (ICSECS-ICOCSIM) , 21-26 August 2021 , Pekan. pp. 200-205.. ISBN 21136432 (Published) http://10.1109/ICSECS52883.2021.00043 DOI: 10.1109/ICSECS52883.2021.00043 |
| spellingShingle | TJ Mechanical engineering and machinery Abu Jafar, Md Muzahid Syafiq Fauzi, Kamarulzaman Md Arafatur, Rahman Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title | Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title_full | Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title_fullStr | Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title_full_unstemmed | Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title_short | Comparison of PPO and SAC Algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| title_sort | comparison of ppo and sac algorithms towards decision making strategies for collision avoidance among multiple autonomous vehicles |
| topic | TJ Mechanical engineering and machinery |
| url | http://umpir.ump.edu.my/id/eprint/33324/1/Comparison_of_PPO_and_SAC_Algorithms_Towards_Decision_Making_Strategies_for_Collision_Avoidance_Among_Multiple_Autonomous_Vehicles%20%281%29.pdf |
| work_keys_str_mv | AT abujafarmdmuzahid comparisonofppoandsacalgorithmstowardsdecisionmakingstrategiesforcollisionavoidanceamongmultipleautonomousvehicles AT syafiqfauzikamarulzaman comparisonofppoandsacalgorithmstowardsdecisionmakingstrategiesforcollisionavoidanceamongmultipleautonomousvehicles AT mdarafaturrahman comparisonofppoandsacalgorithmstowardsdecisionmakingstrategiesforcollisionavoidanceamongmultipleautonomousvehicles |