Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN
Pan-sharpening aims to create high-resolution spectrum images by fusing low-resolution hyperspectral (HS) images with high-resolution panchromatic (PAN) images. Inspired by the Swin transformer used in image classification tasks, this research constructs a three-stream pan-sharpening network based o...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/15/9022 |
| _version_ | 1797587050364403712 |
|---|---|
| author | Zifan Rong Xuesong Jiang Linfeng Huang Hongping Zhou |
| author_facet | Zifan Rong Xuesong Jiang Linfeng Huang Hongping Zhou |
| author_sort | Zifan Rong |
| collection | DOAJ |
| description | Pan-sharpening aims to create high-resolution spectrum images by fusing low-resolution hyperspectral (HS) images with high-resolution panchromatic (PAN) images. Inspired by the Swin transformer used in image classification tasks, this research constructs a three-stream pan-sharpening network based on the Swin transformer and a multi-scale feature extraction module. Unlike the traditional convolutional neural network (CNN) pan-sharpening model, we use the Swin transformer to establish global connections with the image and combine it with a multi-scale feature extraction module to extract local features of different sizes. The model combines the advantages of the Swin transformer and CNN, enabling fused images to maintain good local detail and global linkage by mitigating distortion in hyperspectral images. In order to verify the effectiveness of the method, this paper evaluates fused images with subjective visual and quantitative indicators. Experimental results show that the method proposed in this paper can better preserve the spatial and spectral information of images compared to the classical and latest models. |
| first_indexed | 2024-03-11T00:31:45Z |
| format | Article |
| id | doaj.art-d860e58594374ee790514b6248d4c3bb |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T00:31:45Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-d860e58594374ee790514b6248d4c3bb2023-11-18T22:40:39ZengMDPI AGApplied Sciences2076-34172023-08-011315902210.3390/app13159022Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNNZifan Rong0Xuesong Jiang1Linfeng Huang2Hongping Zhou3School of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing 210037, ChinaSchool of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing 210037, ChinaSchool of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing 210037, ChinaSchool of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing 210037, ChinaPan-sharpening aims to create high-resolution spectrum images by fusing low-resolution hyperspectral (HS) images with high-resolution panchromatic (PAN) images. Inspired by the Swin transformer used in image classification tasks, this research constructs a three-stream pan-sharpening network based on the Swin transformer and a multi-scale feature extraction module. Unlike the traditional convolutional neural network (CNN) pan-sharpening model, we use the Swin transformer to establish global connections with the image and combine it with a multi-scale feature extraction module to extract local features of different sizes. The model combines the advantages of the Swin transformer and CNN, enabling fused images to maintain good local detail and global linkage by mitigating distortion in hyperspectral images. In order to verify the effectiveness of the method, this paper evaluates fused images with subjective visual and quantitative indicators. Experimental results show that the method proposed in this paper can better preserve the spatial and spectral information of images compared to the classical and latest models.https://www.mdpi.com/2076-3417/13/15/9022pan-sharpeningSwin transformermulti-scale residuals and dense blocksresidual feature fusion block |
| spellingShingle | Zifan Rong Xuesong Jiang Linfeng Huang Hongping Zhou Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN Applied Sciences pan-sharpening Swin transformer multi-scale residuals and dense blocks residual feature fusion block |
| title | Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN |
| title_full | Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN |
| title_fullStr | Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN |
| title_full_unstemmed | Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN |
| title_short | Swin–MRDB: Pan-Sharpening Model Based on the Swin Transformer and Multi-Scale CNN |
| title_sort | swin mrdb pan sharpening model based on the swin transformer and multi scale cnn |
| topic | pan-sharpening Swin transformer multi-scale residuals and dense blocks residual feature fusion block |
| url | https://www.mdpi.com/2076-3417/13/15/9022 |
| work_keys_str_mv | AT zifanrong swinmrdbpansharpeningmodelbasedontheswintransformerandmultiscalecnn AT xuesongjiang swinmrdbpansharpeningmodelbasedontheswintransformerandmultiscalecnn AT linfenghuang swinmrdbpansharpeningmodelbasedontheswintransformerandmultiscalecnn AT hongpingzhou swinmrdbpansharpeningmodelbasedontheswintransformerandmultiscalecnn |