SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks
The goal of image fusion is to retain the strengths of different images in the fused result. However, existing fusion algorithms are often complex in design and overlook the influence of attention mechanisms on deep features. To address these issues, we propose an image fusion network based on spati...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/180177 |
| _version_ | 1826110230771007488 |
|---|---|
| author | Wang, Yong Pu, Jianfei Miao, Duoqian Zhang, Longbin Zhang, Lulu Du, Xin |
| author2 | School of Mechanical and Aerospace Engineering |
| author_facet | School of Mechanical and Aerospace Engineering Wang, Yong Pu, Jianfei Miao, Duoqian Zhang, Longbin Zhang, Lulu Du, Xin |
| author_sort | Wang, Yong |
| collection | NTU |
| description | The goal of image fusion is to retain the strengths of different images in the fused result. However, existing fusion algorithms are often complex in design and overlook the influence of attention mechanisms on deep features. To address these issues, we propose an image fusion network based on spatial/channel attention mechanisms and gradient-aggregated residual dense blocks(SCGRFuse). Firstly, we design a novel gradient-aggregated residual dense block (GRXDB) that combines the advantages of ResNeXt and DenseNet, which integrating the Sobel and Laplacian operators to preserve both strong and weak texture features. Then, we introduce spatial and channel attention mechanisms to refine the channel and spatial information of feature maps, enhancing their information capturing capability. Additionally, we leverage a pooling fusion block to merge the refined spatial and channel feature maps, yielding high-quality fusion features. Compared to the existing state-of-the-art methods, experimental results on the MSRS, RoadScene and TNO datasets demonstrate the outstanding fusion performance of our proposed approach. In addition, in the task-driven experiments, SCGRFuse achieved an mIoU accuracy of 71.37%. |
| first_indexed | 2024-10-01T02:31:01Z |
| format | Journal Article |
| id | ntu-10356/180177 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:31:01Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1801772024-09-23T04:35:40Z SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks Wang, Yong Pu, Jianfei Miao, Duoqian Zhang, Longbin Zhang, Lulu Du, Xin School of Mechanical and Aerospace Engineering Rehabilitation Research Institute of Singapore (RRIS) Engineering Image fusion Infrared image The goal of image fusion is to retain the strengths of different images in the fused result. However, existing fusion algorithms are often complex in design and overlook the influence of attention mechanisms on deep features. To address these issues, we propose an image fusion network based on spatial/channel attention mechanisms and gradient-aggregated residual dense blocks(SCGRFuse). Firstly, we design a novel gradient-aggregated residual dense block (GRXDB) that combines the advantages of ResNeXt and DenseNet, which integrating the Sobel and Laplacian operators to preserve both strong and weak texture features. Then, we introduce spatial and channel attention mechanisms to refine the channel and spatial information of feature maps, enhancing their information capturing capability. Additionally, we leverage a pooling fusion block to merge the refined spatial and channel feature maps, yielding high-quality fusion features. Compared to the existing state-of-the-art methods, experimental results on the MSRS, RoadScene and TNO datasets demonstrate the outstanding fusion performance of our proposed approach. In addition, in the task-driven experiments, SCGRFuse achieved an mIoU accuracy of 71.37%. This work is supported by the National Natural Science Foundation of China under Grant 61976158 and Grant 61673301. 2024-09-23T04:35:40Z 2024-09-23T04:35:40Z 2024 Journal Article Wang, Y., Pu, J., Miao, D., Zhang, L., Zhang, L. & Du, X. (2024). SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks. Engineering Applications of Artificial Intelligence, 132, 107898-. https://dx.doi.org/10.1016/j.engappai.2024.107898 0952-1976 https://hdl.handle.net/10356/180177 10.1016/j.engappai.2024.107898 2-s2.0-85184995954 132 107898 en Engineering Applications of Artificial Intelligence © 2024 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering Image fusion Infrared image Wang, Yong Pu, Jianfei Miao, Duoqian Zhang, Longbin Zhang, Lulu Du, Xin SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title | SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title_full | SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title_fullStr | SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title_full_unstemmed | SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title_short | SCGRFuse: an infrared and visible image fusion network based on spatial/channel attention mechanism and gradient aggregation residual dense blocks |
| title_sort | scgrfuse an infrared and visible image fusion network based on spatial channel attention mechanism and gradient aggregation residual dense blocks |
| topic | Engineering Image fusion Infrared image |
| url | https://hdl.handle.net/10356/180177 |
| work_keys_str_mv | AT wangyong scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks AT pujianfei scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks AT miaoduoqian scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks AT zhanglongbin scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks AT zhanglulu scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks AT duxin scgrfuseaninfraredandvisibleimagefusionnetworkbasedonspatialchannelattentionmechanismandgradientaggregationresidualdenseblocks |