Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance
Detecting sparse, small, lost persons with only a few pixels in high-resolution aerial images was, is, and remains an important and difficult mission, in which a vital role is played by accurate monitoring and intelligent co-rescuing for the search and rescue (SaR) system. However, many problems hav...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/11/2928 |
| _version_ | 1797596768077086720 |
|---|---|
| author | Xiangqing Zhang Yan Feng Shun Zhang Nan Wang Shaohui Mei Mingyi He |
| author_facet | Xiangqing Zhang Yan Feng Shun Zhang Nan Wang Shaohui Mei Mingyi He |
| author_sort | Xiangqing Zhang |
| collection | DOAJ |
| description | Detecting sparse, small, lost persons with only a few pixels in high-resolution aerial images was, is, and remains an important and difficult mission, in which a vital role is played by accurate monitoring and intelligent co-rescuing for the search and rescue (SaR) system. However, many problems have not been effectively solved in existing remote-vision-based SaR systems, such as the shortage of person samples in SaR scenarios and the low tolerance of small objects for bounding boxes. To address these issues, a copy-paste mechanism (ISCP) with semi-supervised object detection (SSOD) via instance segmentation and maximum mean discrepancy distance is proposed (MMD), which can provide highly robust, multi-task, and efficient aerial-based person detection for the prototype SaR system. Specifically, numerous pseudo-labels are obtained by accurately segmenting the instances of synthetic ISCP samples to obtain their boundaries. The SSOD trainer then uses soft weights to balance the prediction entropy of the loss function between the ground truth and unreliable labels. Moreover, a novel evaluation metric MMD for anchor-based detectors is proposed to elegantly compute the IoU of the bounding boxes. Extensive experiments and ablation studies on Heridal and optimized public datasets demonstrate that our approach is effective and achieves state-of-the-art person detection performance in aerial images. |
| first_indexed | 2024-03-11T02:57:40Z |
| format | Article |
| id | doaj.art-674babddd4004ef89321d9e5a45be8ac |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T02:57:40Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-674babddd4004ef89321d9e5a45be8ac2023-11-18T08:30:40ZengMDPI AGRemote Sensing2072-42922023-06-011511292810.3390/rs15112928Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy DistanceXiangqing Zhang0Yan Feng1Shun Zhang2Nan Wang3Shaohui Mei4Mingyi He5School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, ChinaDetecting sparse, small, lost persons with only a few pixels in high-resolution aerial images was, is, and remains an important and difficult mission, in which a vital role is played by accurate monitoring and intelligent co-rescuing for the search and rescue (SaR) system. However, many problems have not been effectively solved in existing remote-vision-based SaR systems, such as the shortage of person samples in SaR scenarios and the low tolerance of small objects for bounding boxes. To address these issues, a copy-paste mechanism (ISCP) with semi-supervised object detection (SSOD) via instance segmentation and maximum mean discrepancy distance is proposed (MMD), which can provide highly robust, multi-task, and efficient aerial-based person detection for the prototype SaR system. Specifically, numerous pseudo-labels are obtained by accurately segmenting the instances of synthetic ISCP samples to obtain their boundaries. The SSOD trainer then uses soft weights to balance the prediction entropy of the loss function between the ground truth and unreliable labels. Moreover, a novel evaluation metric MMD for anchor-based detectors is proposed to elegantly compute the IoU of the bounding boxes. Extensive experiments and ablation studies on Heridal and optimized public datasets demonstrate that our approach is effective and achieves state-of-the-art person detection performance in aerial images.https://www.mdpi.com/2072-4292/15/11/2928semi-supervised object detectioncopy-paste mechanisminstance segmentationmaximum mean discrepancyaerial-based person detection |
| spellingShingle | Xiangqing Zhang Yan Feng Shun Zhang Nan Wang Shaohui Mei Mingyi He Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance Remote Sensing semi-supervised object detection copy-paste mechanism instance segmentation maximum mean discrepancy aerial-based person detection |
| title | Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance |
| title_full | Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance |
| title_fullStr | Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance |
| title_full_unstemmed | Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance |
| title_short | Semi-Supervised Person Detection in Aerial Images with Instance Segmentation and Maximum Mean Discrepancy Distance |
| title_sort | semi supervised person detection in aerial images with instance segmentation and maximum mean discrepancy distance |
| topic | semi-supervised object detection copy-paste mechanism instance segmentation maximum mean discrepancy aerial-based person detection |
| url | https://www.mdpi.com/2072-4292/15/11/2928 |
| work_keys_str_mv | AT xiangqingzhang semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance AT yanfeng semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance AT shunzhang semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance AT nanwang semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance AT shaohuimei semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance AT mingyihe semisupervisedpersondetectioninaerialimageswithinstancesegmentationandmaximummeandiscrepancydistance |