Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition
Infrared and visible image fusion (IVIF) aims to generate a fused image with both salient target and rich textures from two different complementary modality images. To better integrate valuable edge information into the fused image, we first propose a novel two-stream network based on Auto-Encoder (...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10424983/ |
| _version_ | 1797305307636957184 |
|---|---|
| author | Haozhe Wang Chang Shu Xiaofeng Li Yu Fu Zhizhong Fu Xiaofeng Yin |
| author_facet | Haozhe Wang Chang Shu Xiaofeng Li Yu Fu Zhizhong Fu Xiaofeng Yin |
| author_sort | Haozhe Wang |
| collection | DOAJ |
| description | Infrared and visible image fusion (IVIF) aims to generate a fused image with both salient target and rich textures from two different complementary modality images. To better integrate valuable edge information into the fused image, we first propose a novel two-stream network based on Auto-Encoder (AE) framework, which extracts deep hierarchical detail information at coarse scale from base stream by multi-level wavelet decomposition progressively and incorporates them into detail stream for information compensation. The aggregation of edge information ranging from coarse to fine facilitates a more comprehensive representation of contours and textures. Then, we propose a new feature fusion strategy, termed as Structural Feature Map Decomposition (SFMD). The first step is to decompose local patches of feature map with each modality into three independent components by Structural Patch Decomposition (SPD). In the second step, appropriate fusion rules are carefully designed for each component and the fused patch can be derived by inverse SPD. Our extensive experiments on several benchmark datasets show that our method outperforms seven compared state-of-the-art methods, especially in human visual perception. |
| first_indexed | 2024-03-08T00:25:02Z |
| format | Article |
| id | doaj.art-944c5d686cee4804b2995573d0e49f5b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T00:25:02Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-944c5d686cee4804b2995573d0e49f5b2024-02-16T00:01:09ZengIEEEIEEE Access2169-35362024-01-0112221902220410.1109/ACCESS.2024.336405010424983Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet DecompositionHaozhe Wang0https://orcid.org/0000-0002-7621-772XChang Shu1https://orcid.org/0009-0003-4972-3082Xiaofeng Li2https://orcid.org/0000-0002-3032-5813Yu Fu3https://orcid.org/0000-0002-5632-8158Zhizhong Fu4https://orcid.org/0000-0002-0072-1073Xiaofeng Yin5School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaLaboratory of Pattern Recognition and Computational Intelligence, Jiangnan University, Wuxi, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaChina International Marine Containers (Group) Company Ltd., Nantong, ChinaInfrared and visible image fusion (IVIF) aims to generate a fused image with both salient target and rich textures from two different complementary modality images. To better integrate valuable edge information into the fused image, we first propose a novel two-stream network based on Auto-Encoder (AE) framework, which extracts deep hierarchical detail information at coarse scale from base stream by multi-level wavelet decomposition progressively and incorporates them into detail stream for information compensation. The aggregation of edge information ranging from coarse to fine facilitates a more comprehensive representation of contours and textures. Then, we propose a new feature fusion strategy, termed as Structural Feature Map Decomposition (SFMD). The first step is to decompose local patches of feature map with each modality into three independent components by Structural Patch Decomposition (SPD). In the second step, appropriate fusion rules are carefully designed for each component and the fused patch can be derived by inverse SPD. Our extensive experiments on several benchmark datasets show that our method outperforms seven compared state-of-the-art methods, especially in human visual perception.https://ieeexplore.ieee.org/document/10424983/Image fusionwavelet decompositionedge informationmulti-scale analysis |
| spellingShingle | Haozhe Wang Chang Shu Xiaofeng Li Yu Fu Zhizhong Fu Xiaofeng Yin Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition IEEE Access Image fusion wavelet decomposition edge information multi-scale analysis |
| title | Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition |
| title_full | Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition |
| title_fullStr | Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition |
| title_full_unstemmed | Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition |
| title_short | Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition |
| title_sort | two stream edge aware network for infrared and visible image fusion with multi level wavelet decomposition |
| topic | Image fusion wavelet decomposition edge information multi-scale analysis |
| url | https://ieeexplore.ieee.org/document/10424983/ |
| work_keys_str_mv | AT haozhewang twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition AT changshu twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition AT xiaofengli twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition AT yufu twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition AT zhizhongfu twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition AT xiaofengyin twostreamedgeawarenetworkforinfraredandvisibleimagefusionwithmultilevelwaveletdecomposition |