CEFusion: Multi‐Modal medical image fusion via cross encoder
Abstract Most existing deep learning‐based multi‐modal medical image fusion (MMIF) methods utilize single‐branch feature extraction strategies to achieve good fusion performance. However, for MMIF tasks, it is thought that this structure cuts off the internal connections between source images, resul...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-10-01
|
| Series: | IET Image Processing |
| Online Access: | https://doi.org/10.1049/ipr2.12549 |
| _version_ | 1811337562655031296 |
|---|---|
| author | Ya Zhu Xue Wang Luping Chen Rencan Nie |
| author_facet | Ya Zhu Xue Wang Luping Chen Rencan Nie |
| author_sort | Ya Zhu |
| collection | DOAJ |
| description | Abstract Most existing deep learning‐based multi‐modal medical image fusion (MMIF) methods utilize single‐branch feature extraction strategies to achieve good fusion performance. However, for MMIF tasks, it is thought that this structure cuts off the internal connections between source images, resulting in information redundancy and degradation of fusion performance. To this end, this paper proposes a novel unsupervised network, termed CEFusion. Different from existing architecture, a cross‐encoder is designed by exploiting the complementary properties between the original image to refine source features through feature interaction and reuse. Furthermore, to force the network to learn complementary information between source images and generate the fused image with high contrast and rich textures, a hybrid loss is proposed consisting of weighted fidelity and gradient losses. Specifically, the weighted fidelity loss can not only force the fusion results to approximate the source images but also effectively preserve the luminance information of the source image through weight estimation, while the gradient loss preserves the texture information of the source image. Experimental results demonstrate the superiority of the method over the state‐of‐the‐art in terms of subjective visual effect and quantitative metrics in various datasets. |
| first_indexed | 2024-04-13T17:56:48Z |
| format | Article |
| id | doaj.art-28aada4eb696429c9a9262a573783cd3 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-04-13T17:56:48Z |
| publishDate | 2022-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-28aada4eb696429c9a9262a573783cd32022-12-22T02:36:27ZengWileyIET Image Processing1751-96591751-96672022-10-0116123177318910.1049/ipr2.12549CEFusion: Multi‐Modal medical image fusion via cross encoderYa Zhu0Xue Wang1Luping Chen2Rencan Nie3School of Information Science and Engineering Yunnan University Kunming 650500 ChinaSchool of Information Science and Engineering Yunnan University Kunming 650500 ChinaSchool of Information Science and Engineering Yunnan University Kunming 650500 ChinaSchool of Information Science and Engineering Yunnan University Kunming 650500 ChinaAbstract Most existing deep learning‐based multi‐modal medical image fusion (MMIF) methods utilize single‐branch feature extraction strategies to achieve good fusion performance. However, for MMIF tasks, it is thought that this structure cuts off the internal connections between source images, resulting in information redundancy and degradation of fusion performance. To this end, this paper proposes a novel unsupervised network, termed CEFusion. Different from existing architecture, a cross‐encoder is designed by exploiting the complementary properties between the original image to refine source features through feature interaction and reuse. Furthermore, to force the network to learn complementary information between source images and generate the fused image with high contrast and rich textures, a hybrid loss is proposed consisting of weighted fidelity and gradient losses. Specifically, the weighted fidelity loss can not only force the fusion results to approximate the source images but also effectively preserve the luminance information of the source image through weight estimation, while the gradient loss preserves the texture information of the source image. Experimental results demonstrate the superiority of the method over the state‐of‐the‐art in terms of subjective visual effect and quantitative metrics in various datasets.https://doi.org/10.1049/ipr2.12549 |
| spellingShingle | Ya Zhu Xue Wang Luping Chen Rencan Nie CEFusion: Multi‐Modal medical image fusion via cross encoder IET Image Processing |
| title | CEFusion: Multi‐Modal medical image fusion via cross encoder |
| title_full | CEFusion: Multi‐Modal medical image fusion via cross encoder |
| title_fullStr | CEFusion: Multi‐Modal medical image fusion via cross encoder |
| title_full_unstemmed | CEFusion: Multi‐Modal medical image fusion via cross encoder |
| title_short | CEFusion: Multi‐Modal medical image fusion via cross encoder |
| title_sort | cefusion multi modal medical image fusion via cross encoder |
| url | https://doi.org/10.1049/ipr2.12549 |
| work_keys_str_mv | AT yazhu cefusionmultimodalmedicalimagefusionviacrossencoder AT xuewang cefusionmultimodalmedicalimagefusionviacrossencoder AT lupingchen cefusionmultimodalmedicalimagefusionviacrossencoder AT rencannie cefusionmultimodalmedicalimagefusionviacrossencoder |