Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN
Convolutional neural networks (CNN) with their deep feature extraction capability have recently been applied in numerous image fusion tasks. However, the image fusion of infrared and visible images leads to loss of fine details and degradation of contrast in the fused image. This deterioration in th...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9044861/ |
| _version_ | 1819171037362782208 |
|---|---|
| author | Lei Yan Jie Cao Saad Rizvi Kaiyu Zhang Qun Hao Xuemin Cheng |
| author_facet | Lei Yan Jie Cao Saad Rizvi Kaiyu Zhang Qun Hao Xuemin Cheng |
| author_sort | Lei Yan |
| collection | DOAJ |
| description | Convolutional neural networks (CNN) with their deep feature extraction capability have recently been applied in numerous image fusion tasks. However, the image fusion of infrared and visible images leads to loss of fine details and degradation of contrast in the fused image. This deterioration in the image is associated with the conventional “averaging” rule for base layer fusion and relatively large feature extraction by CNN. To overcome these problems, an effective fusion framework based on visual saliency weight map (VSWM) combined with CNN is proposed. The proposed framework first employs VSWM method to improve the contrast of an image under consideration. Next, the fine details in the image are preserved by applying multi-resolution singular value decomposition (MSVD) before further processing by CNN. The promising experimental results show that the proposed method outperforms state-of-the-art methods by scoring the highest over different evaluation metrics such as Q0, multiscale structural similarity (MS_SSIM), and the sum of correlations of differences (SCD). |
| first_indexed | 2024-12-22T19:44:55Z |
| format | Article |
| id | doaj.art-6317d979d06d4945a289b78bb764fd9f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T19:44:55Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-6317d979d06d4945a289b78bb764fd9f2022-12-21T18:14:43ZengIEEEIEEE Access2169-35362020-01-018599765998610.1109/ACCESS.2020.29827129044861Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNNLei Yan0Jie Cao1https://orcid.org/0000-0001-8376-7669Saad Rizvi2Kaiyu Zhang3Qun Hao4Xuemin Cheng5Key Laboratory of Biomimetic Robots and Systems, Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Biomimetic Robots and Systems, Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Biomimetic Robots and Systems, Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Biomimetic Robots and Systems, Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Biomimetic Robots and Systems, Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing, ChinaGraduate School at Shenzhen, Tsinghua University, Beijing, ChinaConvolutional neural networks (CNN) with their deep feature extraction capability have recently been applied in numerous image fusion tasks. However, the image fusion of infrared and visible images leads to loss of fine details and degradation of contrast in the fused image. This deterioration in the image is associated with the conventional “averaging” rule for base layer fusion and relatively large feature extraction by CNN. To overcome these problems, an effective fusion framework based on visual saliency weight map (VSWM) combined with CNN is proposed. The proposed framework first employs VSWM method to improve the contrast of an image under consideration. Next, the fine details in the image are preserved by applying multi-resolution singular value decomposition (MSVD) before further processing by CNN. The promising experimental results show that the proposed method outperforms state-of-the-art methods by scoring the highest over different evaluation metrics such as Q0, multiscale structural similarity (MS_SSIM), and the sum of correlations of differences (SCD).https://ieeexplore.ieee.org/document/9044861/Convolutional neural networkimage fusionvisual saliency weight mapmulti-resolution singular value decomposition |
| spellingShingle | Lei Yan Jie Cao Saad Rizvi Kaiyu Zhang Qun Hao Xuemin Cheng Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN IEEE Access Convolutional neural network image fusion visual saliency weight map multi-resolution singular value decomposition |
| title | Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN |
| title_full | Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN |
| title_fullStr | Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN |
| title_full_unstemmed | Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN |
| title_short | Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN |
| title_sort | improving the performance of image fusion based on visual saliency weight map combined with cnn |
| topic | Convolutional neural network image fusion visual saliency weight map multi-resolution singular value decomposition |
| url | https://ieeexplore.ieee.org/document/9044861/ |
| work_keys_str_mv | AT leiyan improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn AT jiecao improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn AT saadrizvi improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn AT kaiyuzhang improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn AT qunhao improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn AT xuemincheng improvingtheperformanceofimagefusionbasedonvisualsaliencyweightmapcombinedwithcnn |