Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN
In order to better preserve the anatomical structure information of Computed Tomography (CT) source images and highlight the metabolic information of lesion regions in Positron Emission Tomography (PET) source images, a hybrid multimodal medical image fusion method (LatLRR-GAN) based on Latent low-r...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/24/12758 |
| _version_ | 1797461660713091072 |
|---|---|
| author | Tao Zhou Qi Li Huiling Lu Xiangxiang Zhang Qianru Cheng |
| author_facet | Tao Zhou Qi Li Huiling Lu Xiangxiang Zhang Qianru Cheng |
| author_sort | Tao Zhou |
| collection | DOAJ |
| description | In order to better preserve the anatomical structure information of Computed Tomography (CT) source images and highlight the metabolic information of lesion regions in Positron Emission Tomography (PET) source images, a hybrid multimodal medical image fusion method (LatLRR-GAN) based on Latent low-rank representation (LatLRR) and the dual discriminators Generative Adversarial Network (ED-D<sup>2</sup>GAN) is proposed. Firstly, considering the denoising capability of LatLRR, source images were decomposed by LatLRR. Secondly, the ED-D<sup>2</sup>GAN model was put forward as the low-rank region fusion method, which can fully extract the information contained by the low-rank region images. Among them, encoder and decoder networks were used in the generator; convolutional neural networks were also used in dual discriminators. Thirdly, a threshold adaptive weighting algorithm based on the region energy ratio is proposed as the salient region fusion rule, which can improve the overall sharpness of the fused image. The experimental results show that compared with the best methods of the other six methods, this paper is effective in multiple objective evaluation metrics, including the average gradient, edge intensity, information entropy, spatial frequency and standard deviation. The results of the two experiments are improved by 35.03%, 42.42%, 4.66%, 8.59% and 11.49% on average. |
| first_indexed | 2024-03-09T17:22:36Z |
| format | Article |
| id | doaj.art-efa6368722924bf28e77627d8cf00776 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T17:22:36Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-efa6368722924bf28e77627d8cf007762023-11-24T13:04:14ZengMDPI AGApplied Sciences2076-34172022-12-0112241275810.3390/app122412758Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GANTao Zhou0Qi Li1Huiling Lu2Xiangxiang Zhang3Qianru Cheng4School of Computer Science and Engineering, North Minzu University, Yinchuan 750021, ChinaSchool of Computer Science and Engineering, North Minzu University, Yinchuan 750021, ChinaSchool of Science, Ningxia Medical University, Yinchuan 750004, ChinaSchool of Computer Science and Engineering, North Minzu University, Yinchuan 750021, ChinaSchool of Computer Science and Engineering, North Minzu University, Yinchuan 750021, ChinaIn order to better preserve the anatomical structure information of Computed Tomography (CT) source images and highlight the metabolic information of lesion regions in Positron Emission Tomography (PET) source images, a hybrid multimodal medical image fusion method (LatLRR-GAN) based on Latent low-rank representation (LatLRR) and the dual discriminators Generative Adversarial Network (ED-D<sup>2</sup>GAN) is proposed. Firstly, considering the denoising capability of LatLRR, source images were decomposed by LatLRR. Secondly, the ED-D<sup>2</sup>GAN model was put forward as the low-rank region fusion method, which can fully extract the information contained by the low-rank region images. Among them, encoder and decoder networks were used in the generator; convolutional neural networks were also used in dual discriminators. Thirdly, a threshold adaptive weighting algorithm based on the region energy ratio is proposed as the salient region fusion rule, which can improve the overall sharpness of the fused image. The experimental results show that compared with the best methods of the other six methods, this paper is effective in multiple objective evaluation metrics, including the average gradient, edge intensity, information entropy, spatial frequency and standard deviation. The results of the two experiments are improved by 35.03%, 42.42%, 4.66%, 8.59% and 11.49% on average.https://www.mdpi.com/2076-3417/12/24/12758multimodalmedical image fusionLatLRRGANdeep learning |
| spellingShingle | Tao Zhou Qi Li Huiling Lu Xiangxiang Zhang Qianru Cheng Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN Applied Sciences multimodal medical image fusion LatLRR GAN deep learning |
| title | Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN |
| title_full | Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN |
| title_fullStr | Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN |
| title_full_unstemmed | Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN |
| title_short | Hybrid Multimodal Medical Image Fusion Method Based on LatLRR and ED-D<sup>2</sup>GAN |
| title_sort | hybrid multimodal medical image fusion method based on latlrr and ed d sup 2 sup gan |
| topic | multimodal medical image fusion LatLRR GAN deep learning |
| url | https://www.mdpi.com/2076-3417/12/24/12758 |
| work_keys_str_mv | AT taozhou hybridmultimodalmedicalimagefusionmethodbasedonlatlrrandeddsup2supgan AT qili hybridmultimodalmedicalimagefusionmethodbasedonlatlrrandeddsup2supgan AT huilinglu hybridmultimodalmedicalimagefusionmethodbasedonlatlrrandeddsup2supgan AT xiangxiangzhang hybridmultimodalmedicalimagefusionmethodbasedonlatlrrandeddsup2supgan AT qianrucheng hybridmultimodalmedicalimagefusionmethodbasedonlatlrrandeddsup2supgan |