MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images
Due to the limitations of current technology and budget, as well as the influence of various factors, obtaining remote sensing images with high-temporal and high-spatial (HTHS) resolution simultaneously is a major challenge. In this paper, we propose the GAN spatiotemporal fusion model Based on mult...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/6/1583 |
| _version_ | 1827747774116921344 |
|---|---|
| author | Hui Liu Guangqi Yang Fengliang Deng Yurong Qian Yingying Fan |
| author_facet | Hui Liu Guangqi Yang Fengliang Deng Yurong Qian Yingying Fan |
| author_sort | Hui Liu |
| collection | DOAJ |
| description | Due to the limitations of current technology and budget, as well as the influence of various factors, obtaining remote sensing images with high-temporal and high-spatial (HTHS) resolution simultaneously is a major challenge. In this paper, we propose the GAN spatiotemporal fusion model Based on multiscale and convolutional block attention module (CBAM) for remote sensing images (MCBAM-GAN) to produce high-quality HTHS fusion images. The model is divided into three stages: multi-level feature extraction, multi-feature fusion, and multi-scale reconstruction. First of all, we use the U-NET structure in the generator to deal with the significant differences in image resolution while avoiding the reduction in resolution due to the limitation of GPU memory. Second, a flexible CBAM module is added to adaptively re-scale the spatial and channel features without increasing the computational cost, to enhance the salient areas and extract more detailed features. Considering that features of different scales play an essential role in the fusion, the idea of multiscale is added to extract features of different scales in different scenes and finally use them in the multi loss reconstruction stage. Finally, to check the validity of MCBAM-GAN model, we test it on LGC and CIA datasets and compare it with the classical algorithm for spatiotemporal fusion. The results show that the model performs well in this paper. |
| first_indexed | 2024-03-11T05:57:01Z |
| format | Article |
| id | doaj.art-2802b8330c0f451eaf61a03dc55ecb8d |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T05:57:01Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-2802b8330c0f451eaf61a03dc55ecb8d2023-11-17T13:39:05ZengMDPI AGRemote Sensing2072-42922023-03-01156158310.3390/rs15061583MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing ImagesHui Liu0Guangqi Yang1Fengliang Deng2Yurong Qian3Yingying Fan4School of Information Science and Engineering, Xinjiang University, Urumqi 830014, ChinaSchool of Software, Xinjiang University, Urumqi 830008, ChinaSchool of Software, Xinjiang University, Urumqi 830008, ChinaKey Laboratory of Signal Detection and Processing in Xinjiang Uygur Autonomous Region, Urumqi 830011, ChinaSchool of Information Science and Engineering, Xinjiang University, Urumqi 830014, ChinaDue to the limitations of current technology and budget, as well as the influence of various factors, obtaining remote sensing images with high-temporal and high-spatial (HTHS) resolution simultaneously is a major challenge. In this paper, we propose the GAN spatiotemporal fusion model Based on multiscale and convolutional block attention module (CBAM) for remote sensing images (MCBAM-GAN) to produce high-quality HTHS fusion images. The model is divided into three stages: multi-level feature extraction, multi-feature fusion, and multi-scale reconstruction. First of all, we use the U-NET structure in the generator to deal with the significant differences in image resolution while avoiding the reduction in resolution due to the limitation of GPU memory. Second, a flexible CBAM module is added to adaptively re-scale the spatial and channel features without increasing the computational cost, to enhance the salient areas and extract more detailed features. Considering that features of different scales play an essential role in the fusion, the idea of multiscale is added to extract features of different scales in different scenes and finally use them in the multi loss reconstruction stage. Finally, to check the validity of MCBAM-GAN model, we test it on LGC and CIA datasets and compare it with the classical algorithm for spatiotemporal fusion. The results show that the model performs well in this paper.https://www.mdpi.com/2072-4292/15/6/1583multi-scaleconvolutional attention modulespatiotemporal fusionremote sensing imagesU-NET |
| spellingShingle | Hui Liu Guangqi Yang Fengliang Deng Yurong Qian Yingying Fan MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images Remote Sensing multi-scale convolutional attention module spatiotemporal fusion remote sensing images U-NET |
| title | MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images |
| title_full | MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images |
| title_fullStr | MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images |
| title_full_unstemmed | MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images |
| title_short | MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images |
| title_sort | mcbam gan the gan spatiotemporal fusion model based on multiscale and cbam for remote sensing images |
| topic | multi-scale convolutional attention module spatiotemporal fusion remote sensing images U-NET |
| url | https://www.mdpi.com/2072-4292/15/6/1583 |
| work_keys_str_mv | AT huiliu mcbamgantheganspatiotemporalfusionmodelbasedonmultiscaleandcbamforremotesensingimages AT guangqiyang mcbamgantheganspatiotemporalfusionmodelbasedonmultiscaleandcbamforremotesensingimages AT fengliangdeng mcbamgantheganspatiotemporalfusionmodelbasedonmultiscaleandcbamforremotesensingimages AT yurongqian mcbamgantheganspatiotemporalfusionmodelbasedonmultiscaleandcbamforremotesensingimages AT yingyingfan mcbamgantheganspatiotemporalfusionmodelbasedonmultiscaleandcbamforremotesensingimages |