SharDif: Sharing and Differential Learning for Image Fusion
Image fusion is the generation of an informative image that contains complementary information from the original sensor images, such as texture details and attentional targets. Existing methods have designed a variety of feature extraction algorithms and fusion strategies to achieve image fusion. Ho...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/26/1/57 |
| _version_ | 1797344087701979136 |
|---|---|
| author | Lei Liang Zhisheng Gao |
| author_facet | Lei Liang Zhisheng Gao |
| author_sort | Lei Liang |
| collection | DOAJ |
| description | Image fusion is the generation of an informative image that contains complementary information from the original sensor images, such as texture details and attentional targets. Existing methods have designed a variety of feature extraction algorithms and fusion strategies to achieve image fusion. However, these methods ignore the extraction of common features in the original multi-source images. The point of view proposed in this paper is that image fusion is to retain, as much as possible, the useful shared features and complementary differential features of the original multi-source images. Shared and differential learning methods for infrared and visible light image fusion are proposed. An encoder with shared weights is used to extract shared common features contained in infrared and visible light images, and the other two encoder blocks are used to extract differential features of infrared images and visible light images, respectively. Effective learning of shared and differential features is achieved through weight sharing and loss functions. Then, the fusion of shared features and differential features is achieved via a weighted fusion strategy based on an entropy-weighted attention mechanism. The experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with the-state-of-the-art methods, the significant advantage of the proposed method is that it retains the structural information of the original image and has better fusion accuracy and visual perception effect. |
| first_indexed | 2024-03-08T10:57:17Z |
| format | Article |
| id | doaj.art-7e8b760057174353ba1ae75157483765 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-08T10:57:17Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-7e8b760057174353ba1ae751574837652024-01-26T16:23:08ZengMDPI AGEntropy1099-43002024-01-012615710.3390/e26010057SharDif: Sharing and Differential Learning for Image FusionLei Liang0Zhisheng Gao1College of Aerospace Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, ChinaSchool of Computer and Software Engineering, Xihua University, Chengdu 610039, ChinaImage fusion is the generation of an informative image that contains complementary information from the original sensor images, such as texture details and attentional targets. Existing methods have designed a variety of feature extraction algorithms and fusion strategies to achieve image fusion. However, these methods ignore the extraction of common features in the original multi-source images. The point of view proposed in this paper is that image fusion is to retain, as much as possible, the useful shared features and complementary differential features of the original multi-source images. Shared and differential learning methods for infrared and visible light image fusion are proposed. An encoder with shared weights is used to extract shared common features contained in infrared and visible light images, and the other two encoder blocks are used to extract differential features of infrared images and visible light images, respectively. Effective learning of shared and differential features is achieved through weight sharing and loss functions. Then, the fusion of shared features and differential features is achieved via a weighted fusion strategy based on an entropy-weighted attention mechanism. The experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with the-state-of-the-art methods, the significant advantage of the proposed method is that it retains the structural information of the original image and has better fusion accuracy and visual perception effect.https://www.mdpi.com/1099-4300/26/1/57image fusionshared featuredifferential featuremulti-level semantic feature |
| spellingShingle | Lei Liang Zhisheng Gao SharDif: Sharing and Differential Learning for Image Fusion Entropy image fusion shared feature differential feature multi-level semantic feature |
| title | SharDif: Sharing and Differential Learning for Image Fusion |
| title_full | SharDif: Sharing and Differential Learning for Image Fusion |
| title_fullStr | SharDif: Sharing and Differential Learning for Image Fusion |
| title_full_unstemmed | SharDif: Sharing and Differential Learning for Image Fusion |
| title_short | SharDif: Sharing and Differential Learning for Image Fusion |
| title_sort | shardif sharing and differential learning for image fusion |
| topic | image fusion shared feature differential feature multi-level semantic feature |
| url | https://www.mdpi.com/1099-4300/26/1/57 |
| work_keys_str_mv | AT leiliang shardifsharinganddifferentiallearningforimagefusion AT zhishenggao shardifsharinganddifferentiallearningforimagefusion |