Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network
The integration of the transformer and convolutional neural network (CNN) has become a useful method for change detection in remote sensing images. The main function of the transformer is to capture the global features, while the CNN is more for obtaining the local features. However, such an integra...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10378644/ |
| _version_ | 1797348786808291328 |
|---|---|
| author | Dalong Zheng Zebin Wu Jia Liu Chih-Cheng Hung Zhihui Wei |
| author_facet | Dalong Zheng Zebin Wu Jia Liu Chih-Cheng Hung Zhihui Wei |
| author_sort | Dalong Zheng |
| collection | DOAJ |
| description | The integration of the transformer and convolutional neural network (CNN) has become a useful method for change detection in remote sensing images. The main function of the transformer is to capture the global features, while the CNN is more for obtaining the local features. However, such an integration is not efficient for change detection in the very-high-resolution (VHR) remote sensing images with fine surface detail information. Hence, to improve this traditional construction of the transformer and CNN, we propose a dense Swin-Transformer-V2 (DST) and VGG16, coined as DST-VGG, for extracting the discriminatory features for change detection. The difference between our proposed network and other networks is that the output of the VGG16 encoders will be used in the DST in which more Swin-V2 blocks are added for fine feature extraction. The learning model in the VGG16 encoders employs a self-supervised method, which is guided through the change in details. Our network not only inherits the advantages of the integration of the transformer and CNN, but also captures the features of change relationship through the DST and catches the primitive features in both prechanged and postchanged regions through the VGG16. In addition, we design a mixed feature pyramid within the DST, which provides interlayer interaction information and intralayer multiscale information for a more complete feature learning within the new network. Furthermore, we impose a self-supervised strategy to guide the VGG16 provide the semantic change information from the output features of the encoder. We compared our experimental results with those of the state-of-the-art methods on four commonly used public VHR remote sensing datasets. It shows that our network performs better, in terms of F1, IoU, and OA, than those of the existing networks for change detection. |
| first_indexed | 2024-03-08T12:11:09Z |
| format | Article |
| id | doaj.art-d17b62cbde434372874df4f424c49674 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-08T12:11:09Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-d17b62cbde434372874df4f424c496742024-01-23T00:00:17ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352024-01-01173181319610.1109/JSTARS.2023.334863010378644Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid NetworkDalong Zheng0https://orcid.org/0000-0003-4572-4558Zebin Wu1https://orcid.org/0000-0002-7162-0202Jia Liu2https://orcid.org/0000-0002-5999-2361Chih-Cheng Hung3https://orcid.org/0000-0003-0477-5957Zhihui Wei4https://orcid.org/0000-0002-4841-6051School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaCenter for Machine Vision and Security Research, Kennesaw State University, Kennesaw, GA, USASchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaThe integration of the transformer and convolutional neural network (CNN) has become a useful method for change detection in remote sensing images. The main function of the transformer is to capture the global features, while the CNN is more for obtaining the local features. However, such an integration is not efficient for change detection in the very-high-resolution (VHR) remote sensing images with fine surface detail information. Hence, to improve this traditional construction of the transformer and CNN, we propose a dense Swin-Transformer-V2 (DST) and VGG16, coined as DST-VGG, for extracting the discriminatory features for change detection. The difference between our proposed network and other networks is that the output of the VGG16 encoders will be used in the DST in which more Swin-V2 blocks are added for fine feature extraction. The learning model in the VGG16 encoders employs a self-supervised method, which is guided through the change in details. Our network not only inherits the advantages of the integration of the transformer and CNN, but also captures the features of change relationship through the DST and catches the primitive features in both prechanged and postchanged regions through the VGG16. In addition, we design a mixed feature pyramid within the DST, which provides interlayer interaction information and intralayer multiscale information for a more complete feature learning within the new network. Furthermore, we impose a self-supervised strategy to guide the VGG16 provide the semantic change information from the output features of the encoder. We compared our experimental results with those of the state-of-the-art methods on four commonly used public VHR remote sensing datasets. It shows that our network performs better, in terms of F1, IoU, and OA, than those of the existing networks for change detection.https://ieeexplore.ieee.org/document/10378644/Change detectionmixed feature pyramid (MFP)self-supervised learning (SSL)Swin transformer V2VGG16 |
| spellingShingle | Dalong Zheng Zebin Wu Jia Liu Chih-Cheng Hung Zhihui Wei Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Change detection mixed feature pyramid (MFP) self-supervised learning (SSL) Swin transformer V2 VGG16 |
| title | Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network |
| title_full | Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network |
| title_fullStr | Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network |
| title_full_unstemmed | Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network |
| title_short | Detail Enhanced Change Detection in VHR Images Using a Self-Supervised Multiscale Hybrid Network |
| title_sort | detail enhanced change detection in vhr images using a self supervised multiscale hybrid network |
| topic | Change detection mixed feature pyramid (MFP) self-supervised learning (SSL) Swin transformer V2 VGG16 |
| url | https://ieeexplore.ieee.org/document/10378644/ |
| work_keys_str_mv | AT dalongzheng detailenhancedchangedetectioninvhrimagesusingaselfsupervisedmultiscalehybridnetwork AT zebinwu detailenhancedchangedetectioninvhrimagesusingaselfsupervisedmultiscalehybridnetwork AT jialiu detailenhancedchangedetectioninvhrimagesusingaselfsupervisedmultiscalehybridnetwork AT chihchenghung detailenhancedchangedetectioninvhrimagesusingaselfsupervisedmultiscalehybridnetwork AT zhihuiwei detailenhancedchangedetectioninvhrimagesusingaselfsupervisedmultiscalehybridnetwork |