Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion
In the field of remote sensing, image fusion technology plays a crucial role in observing the state of global resources and environmental conditions, proposing response strategies, and constantly monitoring and correcting strategies. Currently, the majority of traditional methods exhibit varying deg...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10294268/ |
| _version_ | 1797330390073999360 |
|---|---|
| author | Chuang Liu Lu Wei Zhiqi Zhang Xiaoxiao Feng Shao Xiang |
| author_facet | Chuang Liu Lu Wei Zhiqi Zhang Xiaoxiao Feng Shao Xiang |
| author_sort | Chuang Liu |
| collection | DOAJ |
| description | In the field of remote sensing, image fusion technology plays a crucial role in observing the state of global resources and environmental conditions, proposing response strategies, and constantly monitoring and correcting strategies. Currently, the majority of traditional methods exhibit varying degrees of spatial or spectral distortion, and these unreasonable spectral distributions may contain erroneous geographical feature information. Meanwhile, despite their performance in fusion results, deep learning-based methods cannot be applied to some practical application scenarios due to the requirement for hardware specifications resulting from a large number of parameters in their models. These issues are not conducive to accurately reflecting the actual geomorphic resource conditions or promoting sustainable development. In order to address the above issues, we propose a novel recursive self-attention module (RSAM), which consists of two stages: spatial-spectral similarity extraction and self-attention weight generation. The proposed RSAM employs a global-to-local strategy to capture the global interdependencies of two distinct local locations in the feature map. This method allows for simultaneous consideration of both spatial and spectral information while focusing on more mutual information between spectral and spatial dimensions. Subsequently, we construct the corresponding recursive self-attention residual block (RSARB) through RSAM and concatenate the RSARBs to generate a recursive self-attention module-based network (RSANet) with a limited number of parameters. Extensive experiments demonstrate that RSANet achieves superior results in both qualitative and quantitative evaluation despite the model parameters being within a narrow range of orders of magnitude. This demonstrates that the proposed method possesses robust feature learning capability and practicality for observing and studying the global resource environment. |
| first_indexed | 2024-03-08T07:19:10Z |
| format | Article |
| id | doaj.art-301340ed06964bb4bd7429b6116039f4 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-08T07:19:10Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-301340ed06964bb4bd7429b6116039f42024-02-03T00:01:05ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-0116100671008310.1109/JSTARS.2023.332716710294268Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image FusionChuang Liu0https://orcid.org/0009-0001-8246-3417Lu Wei1https://orcid.org/0009-0006-4363-1672Zhiqi Zhang2https://orcid.org/0000-0003-1914-9430Xiaoxiao Feng3https://orcid.org/0000-0003-3268-7831Shao Xiang4https://orcid.org/0000-0002-2797-1937School of Computer Science, Hubei University of Technology, Wuhan, ChinaSchool of Information Science and Engineering, Wuchang Shouyi University, Wuhan, ChinaSchool of Computer Science, Hubei University of Technology, Wuhan, ChinaSchool of Computer Science, Hubei University of Technology, Wuhan, ChinaState Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, ChinaIn the field of remote sensing, image fusion technology plays a crucial role in observing the state of global resources and environmental conditions, proposing response strategies, and constantly monitoring and correcting strategies. Currently, the majority of traditional methods exhibit varying degrees of spatial or spectral distortion, and these unreasonable spectral distributions may contain erroneous geographical feature information. Meanwhile, despite their performance in fusion results, deep learning-based methods cannot be applied to some practical application scenarios due to the requirement for hardware specifications resulting from a large number of parameters in their models. These issues are not conducive to accurately reflecting the actual geomorphic resource conditions or promoting sustainable development. In order to address the above issues, we propose a novel recursive self-attention module (RSAM), which consists of two stages: spatial-spectral similarity extraction and self-attention weight generation. The proposed RSAM employs a global-to-local strategy to capture the global interdependencies of two distinct local locations in the feature map. This method allows for simultaneous consideration of both spatial and spectral information while focusing on more mutual information between spectral and spatial dimensions. Subsequently, we construct the corresponding recursive self-attention residual block (RSARB) through RSAM and concatenate the RSARBs to generate a recursive self-attention module-based network (RSANet) with a limited number of parameters. Extensive experiments demonstrate that RSANet achieves superior results in both qualitative and quantitative evaluation despite the model parameters being within a narrow range of orders of magnitude. This demonstrates that the proposed method possesses robust feature learning capability and practicality for observing and studying the global resource environment.https://ieeexplore.ieee.org/document/10294268/Earth observationimage fusionremote sensingself-attention |
| spellingShingle | Chuang Liu Lu Wei Zhiqi Zhang Xiaoxiao Feng Shao Xiang Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Earth observation image fusion remote sensing self-attention |
| title | Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion |
| title_full | Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion |
| title_fullStr | Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion |
| title_full_unstemmed | Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion |
| title_short | Recursive Self-Attention Modules-Based Network for Panchromatic and Multispectral Image Fusion |
| title_sort | recursive self attention modules based network for panchromatic and multispectral image fusion |
| topic | Earth observation image fusion remote sensing self-attention |
| url | https://ieeexplore.ieee.org/document/10294268/ |
| work_keys_str_mv | AT chuangliu recursiveselfattentionmodulesbasednetworkforpanchromaticandmultispectralimagefusion AT luwei recursiveselfattentionmodulesbasednetworkforpanchromaticandmultispectralimagefusion AT zhiqizhang recursiveselfattentionmodulesbasednetworkforpanchromaticandmultispectralimagefusion AT xiaoxiaofeng recursiveselfattentionmodulesbasednetworkforpanchromaticandmultispectralimagefusion AT shaoxiang recursiveselfattentionmodulesbasednetworkforpanchromaticandmultispectralimagefusion |