Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)
AI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framewo...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9618926/ |
| _version_ | 1818751636336541696 |
|---|---|
| author | Muhammad Ali Farooq Peter Corcoran Cosmin Rotariu Waseem Shariff |
| author_facet | Muhammad Ali Farooq Peter Corcoran Cosmin Rotariu Waseem Shariff |
| author_sort | Muhammad Ali Farooq |
| collection | DOAJ |
| description | AI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framework for thermal vision with seven key object classes in order to provide superior thermal sensing and scene understanding input for advanced driver-assistance systems (ADAS). The networks are trained on public datasets and is validated on test data with three different test approaches which include test-time augmentation, test-time with no augmentation, and test-time with model ensembling. Additionally, a new model ensemble-based inference engine is proposed, and its efficacy is tested on locally gathered novel test data comprising of 20K thermal frames captured with an uncooled LWIR prototype thermal camera in challenging weather and environmental scenarios. The performance analysis of trained models is investigated by computing precision, recall, and mean average precision scores (mAP). Furthermore, the smaller network variant of thermal-YOLO architecture is optimized using TensorRT inference accelerator, which is then deployed on GPU and resource-constrained edge hardware Nvidia Jetson Nano. This is implemented to explicitly reduce the inference time on GPU as well as on Nvidia Jetson Nano to evaluate the feasibility for added real-time onboard installations. |
| first_indexed | 2024-12-18T04:38:43Z |
| format | Article |
| id | doaj.art-a938dbce487c442981ef89c9254f67b1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-18T04:38:43Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a938dbce487c442981ef89c9254f67b12022-12-21T21:20:46ZengIEEEIEEE Access2169-35362021-01-01915646515648110.1109/ACCESS.2021.31291509618926Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)Muhammad Ali Farooq0https://orcid.org/0000-0003-4116-8021Peter Corcoran1https://orcid.org/0000-0003-1670-4793Cosmin Rotariu2Waseem Shariff3https://orcid.org/0000-0001-7298-9389School of Engineering, National University of Ireland Galway, Galway, IrelandSchool of Engineering, National University of Ireland Galway, Galway, IrelandXperi Corporation, Galway, IrelandSchool of Engineering, National University of Ireland Galway, Galway, IrelandAI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framework for thermal vision with seven key object classes in order to provide superior thermal sensing and scene understanding input for advanced driver-assistance systems (ADAS). The networks are trained on public datasets and is validated on test data with three different test approaches which include test-time augmentation, test-time with no augmentation, and test-time with model ensembling. Additionally, a new model ensemble-based inference engine is proposed, and its efficacy is tested on locally gathered novel test data comprising of 20K thermal frames captured with an uncooled LWIR prototype thermal camera in challenging weather and environmental scenarios. The performance analysis of trained models is investigated by computing precision, recall, and mean average precision scores (mAP). Furthermore, the smaller network variant of thermal-YOLO architecture is optimized using TensorRT inference accelerator, which is then deployed on GPU and resource-constrained edge hardware Nvidia Jetson Nano. This is implemented to explicitly reduce the inference time on GPU as well as on Nvidia Jetson Nano to evaluate the feasibility for added real-time onboard installations.https://ieeexplore.ieee.org/document/9618926/Thermal-infraredobject detectionadvanced driver-assistance systemsdeep learningedge computing |
| spellingShingle | Muhammad Ali Farooq Peter Corcoran Cosmin Rotariu Waseem Shariff Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) IEEE Access Thermal-infrared object detection advanced driver-assistance systems deep learning edge computing |
| title | Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) |
| title_full | Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) |
| title_fullStr | Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) |
| title_full_unstemmed | Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) |
| title_short | Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) |
| title_sort | object detection in thermal spectrum for advanced driver assistance systems adas |
| topic | Thermal-infrared object detection advanced driver-assistance systems deep learning edge computing |
| url | https://ieeexplore.ieee.org/document/9618926/ |
| work_keys_str_mv | AT muhammadalifarooq objectdetectioninthermalspectrumforadvanceddriverassistancesystemsadas AT petercorcoran objectdetectioninthermalspectrumforadvanceddriverassistancesystemsadas AT cosminrotariu objectdetectioninthermalspectrumforadvanceddriverassistancesystemsadas AT waseemshariff objectdetectioninthermalspectrumforadvanceddriverassistancesystemsadas |