Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset
Deep learning (DL) has made tremendous strides in object detection in remote sensing (RS). Still, due to the limitation of small objects' nature (e.g., size, shape, and texture), their detecting performance is less than half that of medium/large objects in closed-set conditions (all test classe...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-03-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843224000025 |
| _version_ | 1797303381425913856 |
|---|---|
| author | Wanjing Zhao Yongxue Liu Peng Liu Huansha Wu Yanzhu Dong |
| author_facet | Wanjing Zhao Yongxue Liu Peng Liu Huansha Wu Yanzhu Dong |
| author_sort | Wanjing Zhao |
| collection | DOAJ |
| description | Deep learning (DL) has made tremendous strides in object detection in remote sensing (RS). Still, due to the limitation of small objects' nature (e.g., size, shape, and texture), their detecting performance is less than half that of medium/large objects in closed-set conditions (all test classes are known in the training phase). In the real world, small-object detection (SOD) over wide areas is far more challenging: models trained with limited positives cannot robustly handle unknown classes (i.e., negatives or backgrounds with similar features to positives), namely, the open-set recognition (OSR) problem. To better perform deep-learning-based SOD, we built a global large-scale training set—flying aircraft (FlyingAC), including ∼3.0 × 105 positives and 2.5 × 105 negatives. We then proposed a series of optimal strategies involving set construction, model training, set volume, model structure, and inference mode. Evaluated based on the FlyingAC, the proposed optimal strategies can mitigate the OSR problem and enhance the inference efficiency. Specifically, we found that (i) training the DL models together with positives and negatives is the most effective way to alleviate the OSR problem (improving the F1 score from 11 % to >90 %). (ii) Optimum training sets volumes exist for DL-based SOD. For the FlyingAC, the optimum volume is 2.0 × 105 positives and 1.5 × 105 negatives. (iii) Optimizing the DL model, such as FPN, attention mechanism, and loss function, can improve the SOD precision. Still, the improvement from the model level is much smaller than the optimization from the dataset level. And (iv) For wide-area detection, the candidate-based inference mode we designed can reduce the inference time to one-fifth and the number of false positives by ∼50 % compared to the commonly used sliding-window inference mode. |
| first_indexed | 2024-03-07T23:52:53Z |
| format | Article |
| id | doaj.art-530487f4e8c7443fa05038ec1790cc49 |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-03-07T23:52:53Z |
| publishDate | 2024-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-530487f4e8c7443fa05038ec1790cc492024-02-19T04:13:06ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322024-03-01127103648Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft datasetWanjing Zhao0Yongxue Liu1Peng Liu2Huansha Wu3Yanzhu Dong4School of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu Province 210023, PR ChinaSchool of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu Province 210023, PR China; Collaborative Innovation Center of South China Sea Studies, Nanjing University, Nanjing, Jiangsu Province 210023, PR China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, Jiangsu Province 210023, PR China; Corresponding author at: Nanjing University, Nanjing, Jiangsu Province 210023, PR China.School of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu Province 210023, PR ChinaSchool of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu Province 210023, PR ChinaSchool of Geography and Ocean Science, Nanjing University, Nanjing, Jiangsu Province 210023, PR China; Collaborative Innovation Center of South China Sea Studies, Nanjing University, Nanjing, Jiangsu Province 210023, PR ChinaDeep learning (DL) has made tremendous strides in object detection in remote sensing (RS). Still, due to the limitation of small objects' nature (e.g., size, shape, and texture), their detecting performance is less than half that of medium/large objects in closed-set conditions (all test classes are known in the training phase). In the real world, small-object detection (SOD) over wide areas is far more challenging: models trained with limited positives cannot robustly handle unknown classes (i.e., negatives or backgrounds with similar features to positives), namely, the open-set recognition (OSR) problem. To better perform deep-learning-based SOD, we built a global large-scale training set—flying aircraft (FlyingAC), including ∼3.0 × 105 positives and 2.5 × 105 negatives. We then proposed a series of optimal strategies involving set construction, model training, set volume, model structure, and inference mode. Evaluated based on the FlyingAC, the proposed optimal strategies can mitigate the OSR problem and enhance the inference efficiency. Specifically, we found that (i) training the DL models together with positives and negatives is the most effective way to alleviate the OSR problem (improving the F1 score from 11 % to >90 %). (ii) Optimum training sets volumes exist for DL-based SOD. For the FlyingAC, the optimum volume is 2.0 × 105 positives and 1.5 × 105 negatives. (iii) Optimizing the DL model, such as FPN, attention mechanism, and loss function, can improve the SOD precision. Still, the improvement from the model level is much smaller than the optimization from the dataset level. And (iv) For wide-area detection, the candidate-based inference mode we designed can reduce the inference time to one-fifth and the number of false positives by ∼50 % compared to the commonly used sliding-window inference mode.http://www.sciencedirect.com/science/article/pii/S1569843224000025Small objectOpen set recognitionRemote sensingDeep learningFlying aircraft detection |
| spellingShingle | Wanjing Zhao Yongxue Liu Peng Liu Huansha Wu Yanzhu Dong Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset International Journal of Applied Earth Observations and Geoinformation Small object Open set recognition Remote sensing Deep learning Flying aircraft detection |
| title | Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset |
| title_full | Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset |
| title_fullStr | Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset |
| title_full_unstemmed | Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset |
| title_short | Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset |
| title_sort | optimal strategies for wide area small object detection using deep learning practices from a global flying aircraft dataset |
| topic | Small object Open set recognition Remote sensing Deep learning Flying aircraft detection |
| url | http://www.sciencedirect.com/science/article/pii/S1569843224000025 |
| work_keys_str_mv | AT wanjingzhao optimalstrategiesforwideareasmallobjectdetectionusingdeeplearningpracticesfromaglobalflyingaircraftdataset AT yongxueliu optimalstrategiesforwideareasmallobjectdetectionusingdeeplearningpracticesfromaglobalflyingaircraftdataset AT pengliu optimalstrategiesforwideareasmallobjectdetectionusingdeeplearningpracticesfromaglobalflyingaircraftdataset AT huanshawu optimalstrategiesforwideareasmallobjectdetectionusingdeeplearningpracticesfromaglobalflyingaircraftdataset AT yanzhudong optimalstrategiesforwideareasmallobjectdetectionusingdeeplearningpracticesfromaglobalflyingaircraftdataset |