Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network
In recent years, there has been increased interest in multispectral pedestrian detection using visible and infrared image pairs. This is due to the complementary visual information provided by these modalities, which enhances the robustness and reliability of pedestrian detection systems. However, c...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-01-01
|
| Series: | Frontiers in Physics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fphy.2023.1322232/full |
| _version_ | 1797350990349860864 |
|---|---|
| author | Li Ma Li Ma Jinjin Wang Jinjin Wang Xinguan Dai Xinguan Dai Hangbiao Gao |
| author_facet | Li Ma Li Ma Jinjin Wang Jinjin Wang Xinguan Dai Xinguan Dai Hangbiao Gao |
| author_sort | Li Ma |
| collection | DOAJ |
| description | In recent years, there has been increased interest in multispectral pedestrian detection using visible and infrared image pairs. This is due to the complementary visual information provided by these modalities, which enhances the robustness and reliability of pedestrian detection systems. However, current research in multispectral pedestrian detection faces the challenge of effectively integrating different modalities to reduce miss rates in the system. This article presents an improved method for multispectral pedestrian detection. The method utilises a saliency detection technique to modify the infrared image and obtain an infrared-enhanced map with clear pedestrian features. Subsequently, a multiscale image features fusion network is designed to efficiently fuse visible and IR-enhanced maps. Finally, the fusion network is supervised by three loss functions for illumination perception, light intensity, and texture information in conjunction with the light perception sub-network. The experimental results demonstrate that the proposed method improves the logarithmic mean miss rate for the three main subgroups (all day, day and night) to 3.12%, 3.06%, and 4.13% respectively, at “reasonable” settings. This is an improvement over the traditional method, which achieved rates of 3.11%, 2.77%, and 2.56% respectively, thus demonstrating the effectiveness of the proposed method. |
| first_indexed | 2024-03-08T12:54:03Z |
| format | Article |
| id | doaj.art-293c7f0969014e36ae8e3a4c8b75b1c7 |
| institution | Directory Open Access Journal |
| issn | 2296-424X |
| language | English |
| last_indexed | 2024-03-08T12:54:03Z |
| publishDate | 2024-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Physics |
| spelling | doaj.art-293c7f0969014e36ae8e3a4c8b75b1c72024-01-19T17:56:25ZengFrontiers Media S.A.Frontiers in Physics2296-424X2024-01-011110.3389/fphy.2023.13222321322232Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion networkLi Ma0Li Ma1Jinjin Wang2Jinjin Wang3Xinguan Dai4Xinguan Dai5Hangbiao Gao6College of Communication and Information Engineering, Xi’an University of Science and Technology, Xi’an, ChinaXi’an Key Laboratory of Heterogeneous Network Convergence Communication, Xi’an, ChinaCollege of Communication and Information Engineering, Xi’an University of Science and Technology, Xi’an, ChinaXi’an Key Laboratory of Heterogeneous Network Convergence Communication, Xi’an, ChinaCollege of Communication and Information Engineering, Xi’an University of Science and Technology, Xi’an, ChinaXi’an Key Laboratory of Heterogeneous Network Convergence Communication, Xi’an, ChinaSafety Supervision Department, Shaanxi Cuijiagou Energy Co., Ltd., Tongchuan, ChinaIn recent years, there has been increased interest in multispectral pedestrian detection using visible and infrared image pairs. This is due to the complementary visual information provided by these modalities, which enhances the robustness and reliability of pedestrian detection systems. However, current research in multispectral pedestrian detection faces the challenge of effectively integrating different modalities to reduce miss rates in the system. This article presents an improved method for multispectral pedestrian detection. The method utilises a saliency detection technique to modify the infrared image and obtain an infrared-enhanced map with clear pedestrian features. Subsequently, a multiscale image features fusion network is designed to efficiently fuse visible and IR-enhanced maps. Finally, the fusion network is supervised by three loss functions for illumination perception, light intensity, and texture information in conjunction with the light perception sub-network. The experimental results demonstrate that the proposed method improves the logarithmic mean miss rate for the three main subgroups (all day, day and night) to 3.12%, 3.06%, and 4.13% respectively, at “reasonable” settings. This is an improvement over the traditional method, which achieved rates of 3.11%, 2.77%, and 2.56% respectively, thus demonstrating the effectiveness of the proposed method.https://www.frontiersin.org/articles/10.3389/fphy.2023.1322232/fullmultispectral pedestrian detectionvisible and infrared imagesaliency detectionmultiscale feature fusionillumination perception |
| spellingShingle | Li Ma Li Ma Jinjin Wang Jinjin Wang Xinguan Dai Xinguan Dai Hangbiao Gao Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network Frontiers in Physics multispectral pedestrian detection visible and infrared image saliency detection multiscale feature fusion illumination perception |
| title | Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network |
| title_full | Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network |
| title_fullStr | Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network |
| title_full_unstemmed | Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network |
| title_short | Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network |
| title_sort | deep saliency detection based pedestrian detection with multispectral multi scale features fusion network |
| topic | multispectral pedestrian detection visible and infrared image saliency detection multiscale feature fusion illumination perception |
| url | https://www.frontiersin.org/articles/10.3389/fphy.2023.1322232/full |
| work_keys_str_mv | AT lima deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT lima deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT jinjinwang deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT jinjinwang deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT xinguandai deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT xinguandai deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork AT hangbiaogao deepsaliencydetectionbasedpedestriandetectionwithmultispectralmultiscalefeaturesfusionnetwork |