A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning
The major challenge faced by autonomous vehicles today is driving through busy roads without getting into an accident, especially with a pedestrian. To avoid collision with pedestrians, the vehicle requires the ability to communicate with a pedestrian to understand their actions. The most challengin...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/25/1/135 |
| _version_ | 1797442985695117312 |
|---|---|
| author | Muhammad Shoaib Farooq Haris Khalid Ansif Arooj Tariq Umer Aamer Bilal Asghar Jawad Rasheed Raed M. Shubair Amani Yahyaoui |
| author_facet | Muhammad Shoaib Farooq Haris Khalid Ansif Arooj Tariq Umer Aamer Bilal Asghar Jawad Rasheed Raed M. Shubair Amani Yahyaoui |
| author_sort | Muhammad Shoaib Farooq |
| collection | DOAJ |
| description | The major challenge faced by autonomous vehicles today is driving through busy roads without getting into an accident, especially with a pedestrian. To avoid collision with pedestrians, the vehicle requires the ability to communicate with a pedestrian to understand their actions. The most challenging task in research on computer vision is to detect pedestrian activities, especially at nighttime. The Advanced Driver-Assistance Systems (ADAS) has been developed for driving and parking support for vehicles to visualize sense, send and receive information from the environment but it lacks to detect nighttime pedestrian actions. This article proposes a framework based on Deep Reinforcement Learning (DRL) using Scale Invariant Faster Region-based Convolutional Neural Networks (SIFRCNN) technologies to efficiently detect pedestrian operations through which the vehicle, as agents train themselves from the environment and are forced to maximize the reward. The SIFRCNN has reduced the running time of detecting pedestrian operations from road images by incorporating Region Proposal Network (RPN) computation. Furthermore, we have used Reinforcement Learning (RL) for optimizing the Q-values and training itself to maximize the reward after getting the state from the SIFRCNN. In addition, the latest incarnation of SIFRCNN achieves near-real-time object detection from road images. The proposed SIFRCNN has been tested on KAIST, City Person, and Caltech datasets. The experimental results show an average improvement of 2.3% miss rate of pedestrian detection at nighttime compared to the other CNN-based pedestrian detectors. |
| first_indexed | 2024-03-09T12:49:34Z |
| format | Article |
| id | doaj.art-fba97fd64d944d45bae4717c4d871ff8 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T12:49:34Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-fba97fd64d944d45bae4717c4d871ff82023-11-30T22:09:10ZengMDPI AGEntropy1099-43002023-01-0125113510.3390/e25010135A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement LearningMuhammad Shoaib Farooq0Haris Khalid1Ansif Arooj2Tariq Umer3Aamer Bilal Asghar4Jawad Rasheed5Raed M. Shubair6Amani Yahyaoui7Department of Computer Science, School of Systems and Technology, University of Management and Technology, Lahore 54000, PakistanDepartment of Computer Science, School of Systems and Technology, University of Management and Technology, Lahore 54000, PakistanDepartment of Information Sciences, Division of Science and Technology, University of Education, Lahore 54000, PakistanDepartment of Computer Science, Lahore Campus, COMSATS University Islamabad, Lahore 54000, PakistanDepartment of Electrical and Computer Engineering, COMSATS University Islamabad, Lahore 54000, PakistanDepartment of Software Engineering, Nisantasi University, Istanbul 34398, TurkeyDepartment of Electrical and Computer Engineering, New York University (NYU), Abu Dhabi 129188, United Arab EmiratesDepartment of Software Engineering, Istanbul Sabahattin Zaim University, Istanbul 34303, TurkeyThe major challenge faced by autonomous vehicles today is driving through busy roads without getting into an accident, especially with a pedestrian. To avoid collision with pedestrians, the vehicle requires the ability to communicate with a pedestrian to understand their actions. The most challenging task in research on computer vision is to detect pedestrian activities, especially at nighttime. The Advanced Driver-Assistance Systems (ADAS) has been developed for driving and parking support for vehicles to visualize sense, send and receive information from the environment but it lacks to detect nighttime pedestrian actions. This article proposes a framework based on Deep Reinforcement Learning (DRL) using Scale Invariant Faster Region-based Convolutional Neural Networks (SIFRCNN) technologies to efficiently detect pedestrian operations through which the vehicle, as agents train themselves from the environment and are forced to maximize the reward. The SIFRCNN has reduced the running time of detecting pedestrian operations from road images by incorporating Region Proposal Network (RPN) computation. Furthermore, we have used Reinforcement Learning (RL) for optimizing the Q-values and training itself to maximize the reward after getting the state from the SIFRCNN. In addition, the latest incarnation of SIFRCNN achieves near-real-time object detection from road images. The proposed SIFRCNN has been tested on KAIST, City Person, and Caltech datasets. The experimental results show an average improvement of 2.3% miss rate of pedestrian detection at nighttime compared to the other CNN-based pedestrian detectors.https://www.mdpi.com/1099-4300/25/1/135deep learningadvanced driving systemneural networkautonomous vehicleintelligent driving systemreinforcement learning |
| spellingShingle | Muhammad Shoaib Farooq Haris Khalid Ansif Arooj Tariq Umer Aamer Bilal Asghar Jawad Rasheed Raed M. Shubair Amani Yahyaoui A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning Entropy deep learning advanced driving system neural network autonomous vehicle intelligent driving system reinforcement learning |
| title | A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning |
| title_full | A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning |
| title_fullStr | A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning |
| title_full_unstemmed | A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning |
| title_short | A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning |
| title_sort | conceptual multi layer framework for the detection of nighttime pedestrian in autonomous vehicles using deep reinforcement learning |
| topic | deep learning advanced driving system neural network autonomous vehicle intelligent driving system reinforcement learning |
| url | https://www.mdpi.com/1099-4300/25/1/135 |
| work_keys_str_mv | AT muhammadshoaibfarooq aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT hariskhalid aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT ansifarooj aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT tariqumer aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT aamerbilalasghar aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT jawadrasheed aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT raedmshubair aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT amaniyahyaoui aconceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT muhammadshoaibfarooq conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT hariskhalid conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT ansifarooj conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT tariqumer conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT aamerbilalasghar conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT jawadrasheed conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT raedmshubair conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning AT amaniyahyaoui conceptualmultilayerframeworkforthedetectionofnighttimepedestrianinautonomousvehiclesusingdeepreinforcementlearning |