Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine
Compared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruis...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/12/5693 |
| _version_ | 1797592680354545664 |
|---|---|
| author | Zhongze Qiu Hangfu Guo Jun Hu Hejun Jiang Chaopeng Luo |
| author_facet | Zhongze Qiu Hangfu Guo Jun Hu Hejun Jiang Chaopeng Luo |
| author_sort | Zhongze Qiu |
| collection | DOAJ |
| description | Compared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruise platform, we establish a multispectral dataset of scatterable mines, with mine-spreading areas of the ground vegetation considered. In order to achieve the robust detection of occluded landmines, first, we employ an active learning strategy to refine the labeling of the multispectral dataset. Then, we propose an image fusion architecture driven by detection, in which we use YOLOv5 for the detection part, to improve the detection performance instructively while enhancing the quality of the fused image. Specifically, a simple and lightweight fusion network is designed to sufficiently aggregate texture details and semantic information of the source images and obtain a higher fusion speed. Moreover, we leverage detection loss as well as a joint-training algorithm to allow the semantic information to dynamically flow back into the fusion network. Extensive qualitative and quantitative experiments demonstrate that the detection-driven fusion (DDF) that we propose can effectively increase the recall rate, especially for occluded landmines, and verify the feasibility of multispectral data through reasonable processing. |
| first_indexed | 2024-03-11T01:57:32Z |
| format | Article |
| id | doaj.art-f52f72617c364073abf16cd1ef7d3e92 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T01:57:32Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-f52f72617c364073abf16cd1ef7d3e922023-11-18T12:34:47ZengMDPI AGSensors1424-82202023-06-012312569310.3390/s23125693Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable LandmineZhongze Qiu0Hangfu Guo1Jun Hu2Hejun Jiang3Chaopeng Luo4School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, ChinaSchool of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, ChinaSchool of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, ChinaScience and Technology on Near-Surface Detection Laboratory, Wuxi 214035, ChinaScience and Technology on Near-Surface Detection Laboratory, Wuxi 214035, ChinaCompared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruise platform, we establish a multispectral dataset of scatterable mines, with mine-spreading areas of the ground vegetation considered. In order to achieve the robust detection of occluded landmines, first, we employ an active learning strategy to refine the labeling of the multispectral dataset. Then, we propose an image fusion architecture driven by detection, in which we use YOLOv5 for the detection part, to improve the detection performance instructively while enhancing the quality of the fused image. Specifically, a simple and lightweight fusion network is designed to sufficiently aggregate texture details and semantic information of the source images and obtain a higher fusion speed. Moreover, we leverage detection loss as well as a joint-training algorithm to allow the semantic information to dynamically flow back into the fusion network. Extensive qualitative and quantitative experiments demonstrate that the detection-driven fusion (DDF) that we propose can effectively increase the recall rate, especially for occluded landmines, and verify the feasibility of multispectral data through reasonable processing.https://www.mdpi.com/1424-8220/23/12/5693deep learninglandmine detectionUAV-bornemultispectral fusionobject occlusion |
| spellingShingle | Zhongze Qiu Hangfu Guo Jun Hu Hejun Jiang Chaopeng Luo Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine Sensors deep learning landmine detection UAV-borne multispectral fusion object occlusion |
| title | Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine |
| title_full | Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine |
| title_fullStr | Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine |
| title_full_unstemmed | Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine |
| title_short | Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine |
| title_sort | joint fusion and detection via deep learning in uav borne multispectral sensing of scatterable landmine |
| topic | deep learning landmine detection UAV-borne multispectral fusion object occlusion |
| url | https://www.mdpi.com/1424-8220/23/12/5693 |
| work_keys_str_mv | AT zhongzeqiu jointfusionanddetectionviadeeplearninginuavbornemultispectralsensingofscatterablelandmine AT hangfuguo jointfusionanddetectionviadeeplearninginuavbornemultispectralsensingofscatterablelandmine AT junhu jointfusionanddetectionviadeeplearninginuavbornemultispectralsensingofscatterablelandmine AT hejunjiang jointfusionanddetectionviadeeplearninginuavbornemultispectralsensingofscatterablelandmine AT chaopengluo jointfusionanddetectionviadeeplearninginuavbornemultispectralsensingofscatterablelandmine |