Hybrid Attention Based Residual Network for Pansharpening
Pansharpening aims at fusing the rich spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack o...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/10/1962 |
| _version_ | 1797533701389680640 |
|---|---|
| author | Qin Liu Letong Han Rui Tan Hongfei Fan Weiqi Li Hongming Zhu Bowen Du Sicong Liu |
| author_facet | Qin Liu Letong Han Rui Tan Hongfei Fan Weiqi Li Hongming Zhu Bowen Du Sicong Liu |
| author_sort | Qin Liu |
| collection | DOAJ |
| description | Pansharpening aims at fusing the rich spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack of spatial detail information, which might prevent the accuracy computation for ground object identification. To alleviate these problems, we propose a Hybrid Attention mechanism-based Residual Neural Network (HARNN). In the proposed network, we develop an encoder attention module in the feature extraction part to better utilize the spectral and spatial features of MS and PAN images. Furthermore, the fusion attention module is designed to alleviate spectral distortion and improve contour details of the fused image. A series of ablation and contrast experiments are conducted on GF-1 and GF-2 datasets. The fusion results with less distorted pixels and more spatial details demonstrate that HARNN can implement the pansharpening task effectively, which outperforms the state-of-the-art algorithms. |
| first_indexed | 2024-03-10T11:18:23Z |
| format | Article |
| id | doaj.art-b86df7f9e20e4a39a0a08f46f99064da |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T11:18:23Z |
| publishDate | 2021-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-b86df7f9e20e4a39a0a08f46f99064da2023-11-21T20:13:38ZengMDPI AGRemote Sensing2072-42922021-05-011310196210.3390/rs13101962Hybrid Attention Based Residual Network for PansharpeningQin Liu0Letong Han1Rui Tan2Hongfei Fan3Weiqi Li4Hongming Zhu5Bowen Du6Sicong Liu7School of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Software Engineering, Tongji University, 4800 Caoan Road Jiading District, Shanghai 201804, ChinaSchool of Geodesy and Geomatics, Tongji University, 1239 Siping Road Yangpu District, Shanghai 200082, ChinaPansharpening aims at fusing the rich spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack of spatial detail information, which might prevent the accuracy computation for ground object identification. To alleviate these problems, we propose a Hybrid Attention mechanism-based Residual Neural Network (HARNN). In the proposed network, we develop an encoder attention module in the feature extraction part to better utilize the spectral and spatial features of MS and PAN images. Furthermore, the fusion attention module is designed to alleviate spectral distortion and improve contour details of the fused image. A series of ablation and contrast experiments are conducted on GF-1 and GF-2 datasets. The fusion results with less distorted pixels and more spatial details demonstrate that HARNN can implement the pansharpening task effectively, which outperforms the state-of-the-art algorithms.https://www.mdpi.com/2072-4292/13/10/1962deep learningHARNNhybrid attention mechanismimage fusionremote sensing |
| spellingShingle | Qin Liu Letong Han Rui Tan Hongfei Fan Weiqi Li Hongming Zhu Bowen Du Sicong Liu Hybrid Attention Based Residual Network for Pansharpening Remote Sensing deep learning HARNN hybrid attention mechanism image fusion remote sensing |
| title | Hybrid Attention Based Residual Network for Pansharpening |
| title_full | Hybrid Attention Based Residual Network for Pansharpening |
| title_fullStr | Hybrid Attention Based Residual Network for Pansharpening |
| title_full_unstemmed | Hybrid Attention Based Residual Network for Pansharpening |
| title_short | Hybrid Attention Based Residual Network for Pansharpening |
| title_sort | hybrid attention based residual network for pansharpening |
| topic | deep learning HARNN hybrid attention mechanism image fusion remote sensing |
| url | https://www.mdpi.com/2072-4292/13/10/1962 |
| work_keys_str_mv | AT qinliu hybridattentionbasedresidualnetworkforpansharpening AT letonghan hybridattentionbasedresidualnetworkforpansharpening AT ruitan hybridattentionbasedresidualnetworkforpansharpening AT hongfeifan hybridattentionbasedresidualnetworkforpansharpening AT weiqili hybridattentionbasedresidualnetworkforpansharpening AT hongmingzhu hybridattentionbasedresidualnetworkforpansharpening AT bowendu hybridattentionbasedresidualnetworkforpansharpening AT sicongliu hybridattentionbasedresidualnetworkforpansharpening |