Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection
<p>Thermal imagery is emerging as a viable candidate for 24-7, all-weather pedestrian detection owning to thermal sensors’ robust performance for pedestrian detection under different weather and illumination conditions. Despite the promising results obtained from combining visible (R...
| Main Authors: | , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
IEEE
2023
|
| _version_ | 1797112021225832448 |
|---|---|
| author | Xie, Q Cheng, T-Y Dai, Z Tran, V Trigoni, A Markham, A |
| author_facet | Xie, Q Cheng, T-Y Dai, Z Tran, V Trigoni, A Markham, A |
| author_sort | Xie, Q |
| collection | OXFORD |
| description | <p>Thermal imagery is emerging as a viable candidate for 24-7, all-weather pedestrian detection owning to thermal sensors’ robust performance for pedestrian detection under different weather and illumination conditions. Despite the promising results obtained from combining visible (RGB) and thermal cameras in multi-spectral fusion techniques, the complex synchronization requirements, including alignment and calibration of sensors, impede their deployment in real-world scenarios. In this paper, we introduce a novel approach for domain adaptation to enhance the performance of pedestrian detection based solely on thermal images. Our proposed approach involves several stages. Firstly, we use both thermal and visible images as input during the training phase. Secondly, we leverage a thermal-to-visible hallucination network to generate feature maps that are similar to those generated by the visible branch. Finally, we design a transformer-based multi-modal fusion module to integrate the hallucinated visible and thermal information more effectively. The thermal-to-visible hallucination network acts as domain adaptation, allowing us to obtain pseudo-visual and thermal features using solely thermal input. Based on the experimental results, it is observed the mean average precision (mAP) increases by 4.72% and the miss rate decreases by 7.56% on the KAIST dataset when compared to the baseline model.</p> |
| first_indexed | 2024-03-07T08:18:27Z |
| format | Journal article |
| id | oxford-uuid:419524a7-d50d-4983-a2f2-3472951ac614 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:18:27Z |
| publishDate | 2023 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:419524a7-d50d-4983-a2f2-3472951ac6142024-01-19T07:03:55ZIllumination-aware hallucination-based domain adaptation for thermal pedestrian detectionJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:419524a7-d50d-4983-a2f2-3472951ac614EnglishSymplectic ElementsIEEE2023Xie, QCheng, T-YDai, ZTran, VTrigoni, AMarkham, A<p>Thermal imagery is emerging as a viable candidate for 24-7, all-weather pedestrian detection owning to thermal sensors’ robust performance for pedestrian detection under different weather and illumination conditions. Despite the promising results obtained from combining visible (RGB) and thermal cameras in multi-spectral fusion techniques, the complex synchronization requirements, including alignment and calibration of sensors, impede their deployment in real-world scenarios. In this paper, we introduce a novel approach for domain adaptation to enhance the performance of pedestrian detection based solely on thermal images. Our proposed approach involves several stages. Firstly, we use both thermal and visible images as input during the training phase. Secondly, we leverage a thermal-to-visible hallucination network to generate feature maps that are similar to those generated by the visible branch. Finally, we design a transformer-based multi-modal fusion module to integrate the hallucinated visible and thermal information more effectively. The thermal-to-visible hallucination network acts as domain adaptation, allowing us to obtain pseudo-visual and thermal features using solely thermal input. Based on the experimental results, it is observed the mean average precision (mAP) increases by 4.72% and the miss rate decreases by 7.56% on the KAIST dataset when compared to the baseline model.</p> |
| spellingShingle | Xie, Q Cheng, T-Y Dai, Z Tran, V Trigoni, A Markham, A Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title | Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title_full | Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title_fullStr | Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title_full_unstemmed | Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title_short | Illumination-aware hallucination-based domain adaptation for thermal pedestrian detection |
| title_sort | illumination aware hallucination based domain adaptation for thermal pedestrian detection |
| work_keys_str_mv | AT xieq illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection AT chengty illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection AT daiz illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection AT tranv illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection AT trigonia illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection AT markhama illuminationawarehallucinationbaseddomainadaptationforthermalpedestriandetection |