Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution
In recent years, deep convolutional neural networks with multi-scale features have been widely used in image super-resolution reconstruction (ISR), and the quality of the generated images has been significantly improved compared with traditional methods. However, in current image super-resolution ne...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/6/2634 |
| _version_ | 1797242150507773952 |
|---|---|
| author | Yimin Ma Yi Xu Yunqing Liu Fei Yan Qiong Zhang Qi Li Quanyang Liu |
| author_facet | Yimin Ma Yi Xu Yunqing Liu Fei Yan Qiong Zhang Qi Li Quanyang Liu |
| author_sort | Yimin Ma |
| collection | DOAJ |
| description | In recent years, deep convolutional neural networks with multi-scale features have been widely used in image super-resolution reconstruction (ISR), and the quality of the generated images has been significantly improved compared with traditional methods. However, in current image super-resolution network algorithms, these methods need to be further explored in terms of the effective fusion of multi-scale features and cross-domain application of attention mechanisms. To address these issues, we propose a novel multi-scale cross-attention fusion network (MCFN), which optimizes the feature extraction and fusion process in structural design and modular innovation. In order to make better use of the attention mechanism, we propose a Pyramid Multi-scale Module (PMM) to extract multi-scale information by cascading. This PMM is introduced in MCFN and is mainly constructed by multiple multi-scale cross-attention modules (MTMs). To fuse the feature information of PMMs efficiently in both channel and spatial dimensions, we propose the cross-attention fusion module (CFM). In addition, an improved integrated attention enhancement module (IAEM) is inserted at the network’s end to enhance the correlation of high-frequency feature information between layers. Experimental results show that the algorithm significantly improves the reconstructed images’ edge information and texture details, and the benchmark dataset’s performance evaluation shows comparable performance to current state-of-the-art techniques. |
| first_indexed | 2024-04-24T18:34:39Z |
| format | Article |
| id | doaj.art-9801b525c8f844c4bfcc541a2b9325a8 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-04-24T18:34:39Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-9801b525c8f844c4bfcc541a2b9325a82024-03-27T13:20:19ZengMDPI AGApplied Sciences2076-34172024-03-01146263410.3390/app14062634Multi-Scale Cross-Attention Fusion Network Based on Image Super-ResolutionYimin Ma0Yi Xu1Yunqing Liu2Fei Yan3Qiong Zhang4Qi Li5Quanyang Liu6The College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaThe College of Electronic and Information Engineering, Changchun University of Science and Technology, Changchun 130022, ChinaIn recent years, deep convolutional neural networks with multi-scale features have been widely used in image super-resolution reconstruction (ISR), and the quality of the generated images has been significantly improved compared with traditional methods. However, in current image super-resolution network algorithms, these methods need to be further explored in terms of the effective fusion of multi-scale features and cross-domain application of attention mechanisms. To address these issues, we propose a novel multi-scale cross-attention fusion network (MCFN), which optimizes the feature extraction and fusion process in structural design and modular innovation. In order to make better use of the attention mechanism, we propose a Pyramid Multi-scale Module (PMM) to extract multi-scale information by cascading. This PMM is introduced in MCFN and is mainly constructed by multiple multi-scale cross-attention modules (MTMs). To fuse the feature information of PMMs efficiently in both channel and spatial dimensions, we propose the cross-attention fusion module (CFM). In addition, an improved integrated attention enhancement module (IAEM) is inserted at the network’s end to enhance the correlation of high-frequency feature information between layers. Experimental results show that the algorithm significantly improves the reconstructed images’ edge information and texture details, and the benchmark dataset’s performance evaluation shows comparable performance to current state-of-the-art techniques.https://www.mdpi.com/2076-3417/14/6/2634image super-resolutionpyramid multi-scale featuresinter-attention mechanism |
| spellingShingle | Yimin Ma Yi Xu Yunqing Liu Fei Yan Qiong Zhang Qi Li Quanyang Liu Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution Applied Sciences image super-resolution pyramid multi-scale features inter-attention mechanism |
| title | Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution |
| title_full | Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution |
| title_fullStr | Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution |
| title_full_unstemmed | Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution |
| title_short | Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution |
| title_sort | multi scale cross attention fusion network based on image super resolution |
| topic | image super-resolution pyramid multi-scale features inter-attention mechanism |
| url | https://www.mdpi.com/2076-3417/14/6/2634 |
| work_keys_str_mv | AT yiminma multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT yixu multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT yunqingliu multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT feiyan multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT qiongzhang multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT qili multiscalecrossattentionfusionnetworkbasedonimagesuperresolution AT quanyangliu multiscalecrossattentionfusionnetworkbasedonimagesuperresolution |