Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System
Pedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector sho...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/10/8/934 |
| _version_ | 1797537750971318272 |
|---|---|
| author | Paulius Tumas Artūras Serackis Adam Nowosielski |
| author_facet | Paulius Tumas Artūras Serackis Adam Nowosielski |
| author_sort | Paulius Tumas |
| collection | DOAJ |
| description | Pedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector should work in severe weather conditions. However, only a few datasets include some examples of far-infrared images with distortions caused by atmospheric precipitation and dirt covering sensor optics. This paper proposes the deep learning-based data augmentation technique to enrich far-infrared images collected in good weather conditions by distortions, similar to those caused by bad weather. The six most accurate and fast detectors (TinyV3, TinyL3, You Only Look Once (YOLO)v3, YOLOv4, ResNet50, and ResNext50), performing faster than 15 FPS, were trained on 207,001 annotations and tested on 156,345 annotations, not used for training. The proposed data augmentation technique showed up to a 9.38 mean Average Precision (mAP) increase of pedestrian detection with a maximum of 87.02 mAP (YOLOv4). Proposed in this paper detectors’ Head modifications based on a confidence heat-map gave an additional boost of precision for all six detectors. The most accurate current detector, based on YOLOv4, reached up to 87.20 mAP during our experimental tests. |
| first_indexed | 2024-03-10T12:20:40Z |
| format | Article |
| id | doaj.art-dd3dadfe25b3442d9fbabcc1574b7251 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T12:20:40Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-dd3dadfe25b3442d9fbabcc1574b72512023-11-21T15:33:39ZengMDPI AGElectronics2079-92922021-04-0110893410.3390/electronics10080934Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection SystemPaulius Tumas0Artūras Serackis1Adam Nowosielski2Department of Electronic Systems, Vilnius Gediminas Technical University, 10223 Vilnius, LithuaniaDepartment of Electronic Systems, Vilnius Gediminas Technical University, 10223 Vilnius, LithuaniaFaculty of Computer Science and Information Technology, West Pomeranian University of Technology, 71-210 Szczecin, PolandPedestrian detection is an essential task for computer vision and the automotive industry. Complex systems like advanced driver-assistance systems are based on far-infrared data sensors, used to detect pedestrians at nighttime, fog, rain, and direct sun situations. The robust pedestrian detector should work in severe weather conditions. However, only a few datasets include some examples of far-infrared images with distortions caused by atmospheric precipitation and dirt covering sensor optics. This paper proposes the deep learning-based data augmentation technique to enrich far-infrared images collected in good weather conditions by distortions, similar to those caused by bad weather. The six most accurate and fast detectors (TinyV3, TinyL3, You Only Look Once (YOLO)v3, YOLOv4, ResNet50, and ResNext50), performing faster than 15 FPS, were trained on 207,001 annotations and tested on 156,345 annotations, not used for training. The proposed data augmentation technique showed up to a 9.38 mean Average Precision (mAP) increase of pedestrian detection with a maximum of 87.02 mAP (YOLOv4). Proposed in this paper detectors’ Head modifications based on a confidence heat-map gave an additional boost of precision for all six detectors. The most accurate current detector, based on YOLOv4, reached up to 87.20 mAP during our experimental tests.https://www.mdpi.com/2079-9292/10/8/934FIR pedestrian detectionimage noisedata augmentationbad weatherconfidense heat-mapADAS |
| spellingShingle | Paulius Tumas Artūras Serackis Adam Nowosielski Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System Electronics FIR pedestrian detection image noise data augmentation bad weather confidense heat-map ADAS |
| title | Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System |
| title_full | Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System |
| title_fullStr | Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System |
| title_full_unstemmed | Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System |
| title_short | Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System |
| title_sort | augmentation of severe weather impact to far infrared sensor images to improve pedestrian detection system |
| topic | FIR pedestrian detection image noise data augmentation bad weather confidense heat-map ADAS |
| url | https://www.mdpi.com/2079-9292/10/8/934 |
| work_keys_str_mv | AT pauliustumas augmentationofsevereweatherimpacttofarinfraredsensorimagestoimprovepedestriandetectionsystem AT arturasserackis augmentationofsevereweatherimpacttofarinfraredsensorimagestoimprovepedestriandetectionsystem AT adamnowosielski augmentationofsevereweatherimpacttofarinfraredsensorimagestoimprovepedestriandetectionsystem |