CycleGAN-Based SAR-Optical Image Fusion for Target Recognition
The efficiency and accuracy of target recognition in synthetic aperture radar (SAR) imagery have seen significant progress lately, stemming from the encouraging advancements of automatic target recognition (ATR) technology based on deep learning. However, the development of a deep learning-based SAR...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/23/5569 |
| _version_ | 1827592121075367936 |
|---|---|
| author | Yuchuang Sun Kaijia Yan Wangzhe Li |
| author_facet | Yuchuang Sun Kaijia Yan Wangzhe Li |
| author_sort | Yuchuang Sun |
| collection | DOAJ |
| description | The efficiency and accuracy of target recognition in synthetic aperture radar (SAR) imagery have seen significant progress lately, stemming from the encouraging advancements of automatic target recognition (ATR) technology based on deep learning. However, the development of a deep learning-based SAR ATR algorithm still faces two critical challenges: the difficulty of feature extraction caused by the unique nature of SAR imagery and the scarcity of datasets caused by the high acquisition cost. Due to its desirable image nature and extremely low acquisition cost, the simulated optical target imagery obtained through computer simulation is considered a valuable complement to SAR imagery. In this study, a CycleGAN-based SAR and simulated optical image fusion network (SOIF-CycleGAN) is designed and demonstrated to mitigate the adverse effects of both challenges simultaneously through SAR-optical image bidirectional translation. SAR-to-optical (S2O) image translation produces artificial optical images that are high-quality and rich in details, which are used as supplementary information for SAR images to assist ATR. Conversely, optical-to-SAR (O2S) image translation generates pattern-rich artificial SAR images and provides additional training data for SAR ATR algorithms. Meanwhile, a new dataset of SAR-optical image pairs containing eight different types of aircraft has been created for training and testing SOIF-CycleGAN. By combining image-quality assessment (IQA) methods and human vision, the evaluation verified that the proposed network possesses exceptional bidirectional translation capability. Finally, the results of the S2O and O2S image translations are simultaneously integrated into a SAR ATR network, resulting in an overall accuracy improvement of 6.33%. This demonstrates the effectiveness of SAR-optical image fusion in enhancing the performance of SAR ATR. |
| first_indexed | 2024-03-09T01:43:16Z |
| format | Article |
| id | doaj.art-a402b27f5c5a47a59cee864ebfb23a16 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T01:43:16Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-a402b27f5c5a47a59cee864ebfb23a162023-12-08T15:25:04ZengMDPI AGRemote Sensing2072-42922023-11-011523556910.3390/rs15235569CycleGAN-Based SAR-Optical Image Fusion for Target RecognitionYuchuang Sun0Kaijia Yan1Wangzhe Li2National Key Laboratory of Microwave Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaNational Key Laboratory of Microwave Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaNational Key Laboratory of Microwave Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, ChinaThe efficiency and accuracy of target recognition in synthetic aperture radar (SAR) imagery have seen significant progress lately, stemming from the encouraging advancements of automatic target recognition (ATR) technology based on deep learning. However, the development of a deep learning-based SAR ATR algorithm still faces two critical challenges: the difficulty of feature extraction caused by the unique nature of SAR imagery and the scarcity of datasets caused by the high acquisition cost. Due to its desirable image nature and extremely low acquisition cost, the simulated optical target imagery obtained through computer simulation is considered a valuable complement to SAR imagery. In this study, a CycleGAN-based SAR and simulated optical image fusion network (SOIF-CycleGAN) is designed and demonstrated to mitigate the adverse effects of both challenges simultaneously through SAR-optical image bidirectional translation. SAR-to-optical (S2O) image translation produces artificial optical images that are high-quality and rich in details, which are used as supplementary information for SAR images to assist ATR. Conversely, optical-to-SAR (O2S) image translation generates pattern-rich artificial SAR images and provides additional training data for SAR ATR algorithms. Meanwhile, a new dataset of SAR-optical image pairs containing eight different types of aircraft has been created for training and testing SOIF-CycleGAN. By combining image-quality assessment (IQA) methods and human vision, the evaluation verified that the proposed network possesses exceptional bidirectional translation capability. Finally, the results of the S2O and O2S image translations are simultaneously integrated into a SAR ATR network, resulting in an overall accuracy improvement of 6.33%. This demonstrates the effectiveness of SAR-optical image fusion in enhancing the performance of SAR ATR.https://www.mdpi.com/2072-4292/15/23/5569synthetic aperture radar (SAR)automatic target recognition (ATR)image fusiondeep learningCycleGAN |
| spellingShingle | Yuchuang Sun Kaijia Yan Wangzhe Li CycleGAN-Based SAR-Optical Image Fusion for Target Recognition Remote Sensing synthetic aperture radar (SAR) automatic target recognition (ATR) image fusion deep learning CycleGAN |
| title | CycleGAN-Based SAR-Optical Image Fusion for Target Recognition |
| title_full | CycleGAN-Based SAR-Optical Image Fusion for Target Recognition |
| title_fullStr | CycleGAN-Based SAR-Optical Image Fusion for Target Recognition |
| title_full_unstemmed | CycleGAN-Based SAR-Optical Image Fusion for Target Recognition |
| title_short | CycleGAN-Based SAR-Optical Image Fusion for Target Recognition |
| title_sort | cyclegan based sar optical image fusion for target recognition |
| topic | synthetic aperture radar (SAR) automatic target recognition (ATR) image fusion deep learning CycleGAN |
| url | https://www.mdpi.com/2072-4292/15/23/5569 |
| work_keys_str_mv | AT yuchuangsun cycleganbasedsaropticalimagefusionfortargetrecognition AT kaijiayan cycleganbasedsaropticalimagefusionfortargetrecognition AT wangzheli cycleganbasedsaropticalimagefusionfortargetrecognition |