A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images
Accurate multi-scale object detection in remote sensing images poses a challenge due to the complexity of transferring deep features to shallow features among multi-scale objects. Therefore, this study developed a multi-feature fusion and attention network (MFANet) based on YOLOX. By reparameterizin...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/8/2096 |
| _version_ | 1797603567267217408 |
|---|---|
| author | Yong Cheng Wei Wang Wenjie Zhang Ling Yang Jun Wang Huan Ni Tingzhao Guan Jiaxin He Yakang Gu Ngoc Nguyen Tran |
| author_facet | Yong Cheng Wei Wang Wenjie Zhang Ling Yang Jun Wang Huan Ni Tingzhao Guan Jiaxin He Yakang Gu Ngoc Nguyen Tran |
| author_sort | Yong Cheng |
| collection | DOAJ |
| description | Accurate multi-scale object detection in remote sensing images poses a challenge due to the complexity of transferring deep features to shallow features among multi-scale objects. Therefore, this study developed a multi-feature fusion and attention network (MFANet) based on YOLOX. By reparameterizing the backbone, fusing multi-branch convolution and attention mechanisms, and optimizing the loss function, the MFANet strengthened the feature extraction of objects at different sizes and increased the detection accuracy. The ablation experiment was carried out on the NWPU VHR-10 dataset. Our results showed that the overall performance of the improved network was around 2.94% higher than the average performance of every single module. Based on the comparison experiments, the improved MFANet demonstrated a high mean average precision of 98.78% for 9 classes of objects in the NWPU VHR-10 10-class detection dataset and 94.91% for 11 classes in the DIOR 20-class detection dataset. Overall, MFANet achieved an mAP of 96.63% and 87.88% acting on the NWPU VHR-10 and DIOR datasets, respectively. This method can promote the development of multi-scale object detection in remote sensing images and has the potential to serve and expand intelligent system research in related fields such as object tracking, semantic segmentation, and scene understanding. |
| first_indexed | 2024-03-11T04:33:52Z |
| format | Article |
| id | doaj.art-467d25d0cb6c4c0f91d6066d0cc78971 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T04:33:52Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-467d25d0cb6c4c0f91d6066d0cc789712023-11-17T21:11:58ZengMDPI AGRemote Sensing2072-42922023-04-01158209610.3390/rs15082096A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing ImagesYong Cheng0Wei Wang1Wenjie Zhang2Ling Yang3Jun Wang4Huan Ni5Tingzhao Guan6Jiaxin He7Yakang Gu8Ngoc Nguyen Tran9School of Software, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Automation, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Geographical Sciences, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Software, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Software, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Software, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Automation, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Software, Nanjing University of Information Science & Technology, Nanjing 210044, ChinaSchool of Information and Communication Technology, Hanoi University of Science and Technology, Hanoi 100803, VietnamAccurate multi-scale object detection in remote sensing images poses a challenge due to the complexity of transferring deep features to shallow features among multi-scale objects. Therefore, this study developed a multi-feature fusion and attention network (MFANet) based on YOLOX. By reparameterizing the backbone, fusing multi-branch convolution and attention mechanisms, and optimizing the loss function, the MFANet strengthened the feature extraction of objects at different sizes and increased the detection accuracy. The ablation experiment was carried out on the NWPU VHR-10 dataset. Our results showed that the overall performance of the improved network was around 2.94% higher than the average performance of every single module. Based on the comparison experiments, the improved MFANet demonstrated a high mean average precision of 98.78% for 9 classes of objects in the NWPU VHR-10 10-class detection dataset and 94.91% for 11 classes in the DIOR 20-class detection dataset. Overall, MFANet achieved an mAP of 96.63% and 87.88% acting on the NWPU VHR-10 and DIOR datasets, respectively. This method can promote the development of multi-scale object detection in remote sensing images and has the potential to serve and expand intelligent system research in related fields such as object tracking, semantic segmentation, and scene understanding.https://www.mdpi.com/2072-4292/15/8/2096remote sensing imagesmulti-scale object detectionmulti-feature fusion and attention networkmulti-branch convolutionattention mechanismloss function |
| spellingShingle | Yong Cheng Wei Wang Wenjie Zhang Ling Yang Jun Wang Huan Ni Tingzhao Guan Jiaxin He Yakang Gu Ngoc Nguyen Tran A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images Remote Sensing remote sensing images multi-scale object detection multi-feature fusion and attention network multi-branch convolution attention mechanism loss function |
| title | A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images |
| title_full | A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images |
| title_fullStr | A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images |
| title_full_unstemmed | A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images |
| title_short | A Multi-Feature Fusion and Attention Network for Multi-Scale Object Detection in Remote Sensing Images |
| title_sort | multi feature fusion and attention network for multi scale object detection in remote sensing images |
| topic | remote sensing images multi-scale object detection multi-feature fusion and attention network multi-branch convolution attention mechanism loss function |
| url | https://www.mdpi.com/2072-4292/15/8/2096 |
| work_keys_str_mv | AT yongcheng amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT weiwang amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT wenjiezhang amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT lingyang amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT junwang amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT huanni amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT tingzhaoguan amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT jiaxinhe amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT yakanggu amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT ngocnguyentran amultifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT yongcheng multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT weiwang multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT wenjiezhang multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT lingyang multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT junwang multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT huanni multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT tingzhaoguan multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT jiaxinhe multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT yakanggu multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages AT ngocnguyentran multifeaturefusionandattentionnetworkformultiscaleobjectdetectioninremotesensingimages |