Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space
In order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of i...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/22/12/1423 |
| _version_ | 1797544376589615104 |
|---|---|
| author | Kai Guo Xiongfei Li Hongrui Zang Tiehu Fan |
| author_facet | Kai Guo Xiongfei Li Hongrui Zang Tiehu Fan |
| author_sort | Kai Guo |
| collection | DOAJ |
| description | In order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of intuitive fuzzy processing (IFP), capture image details network (CIDN), fusion, and decoding. First, the membership function of the image is redefined to remove redundant features and obtain the image with complete features. Then, inspired by DenseNet, we proposed a new encoder to capture all the medical information features in the source image. In the fusion layer, we calculate the weight of each feature graph in the required fusion coefficient according to the trajectory of the feature graph. Finally, the filtered medical information is spliced and decoded to reproduce the required fusion image. In the encoding and image reconstruction networks, the mixed loss function of cross entropy and structural similarity is adopted to greatly reduce the information loss in image fusion. To assess performance, we conducted three sets of experiments on medical images of different grayscales and colors. Experimental results show that the proposed algorithm has advantages not only in detail and structure recognition but also in visual features and time complexity compared with other algorithms. |
| first_indexed | 2024-03-10T13:59:36Z |
| format | Article |
| id | doaj.art-eb59475bb26a40ecb9a94388c11ee193 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T13:59:36Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-eb59475bb26a40ecb9a94388c11ee1932023-11-21T01:22:07ZengMDPI AGEntropy1099-43002020-12-012212142310.3390/e22121423Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color SpaceKai Guo0Xiongfei Li1Hongrui Zang2Tiehu Fan3Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, ChinaKey Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, ChinaInformation and Communication Company, State Grid Jilin Electric Power Co., Ltd., Changchun 130022, ChinaCollege of Instrumentation and Electrical Engineering, Jilin University, Changchun 130012, ChinaIn order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of intuitive fuzzy processing (IFP), capture image details network (CIDN), fusion, and decoding. First, the membership function of the image is redefined to remove redundant features and obtain the image with complete features. Then, inspired by DenseNet, we proposed a new encoder to capture all the medical information features in the source image. In the fusion layer, we calculate the weight of each feature graph in the required fusion coefficient according to the trajectory of the feature graph. Finally, the filtered medical information is spliced and decoded to reproduce the required fusion image. In the encoding and image reconstruction networks, the mixed loss function of cross entropy and structural similarity is adopted to greatly reduce the information loss in image fusion. To assess performance, we conducted three sets of experiments on medical images of different grayscales and colors. Experimental results show that the proposed algorithm has advantages not only in detail and structure recognition but also in visual features and time complexity compared with other algorithms.https://www.mdpi.com/1099-4300/22/12/1423intuitive fuzzy processingcapture image details networkSeLU activation functiontrace of a feature mapimage entropy and cross entropyYIQ color space |
| spellingShingle | Kai Guo Xiongfei Li Hongrui Zang Tiehu Fan Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space Entropy intuitive fuzzy processing capture image details network SeLU activation function trace of a feature map image entropy and cross entropy YIQ color space |
| title | Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space |
| title_full | Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space |
| title_fullStr | Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space |
| title_full_unstemmed | Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space |
| title_short | Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space |
| title_sort | multi modal medical image fusion based on fusionnet in yiq color space |
| topic | intuitive fuzzy processing capture image details network SeLU activation function trace of a feature map image entropy and cross entropy YIQ color space |
| url | https://www.mdpi.com/1099-4300/22/12/1423 |
| work_keys_str_mv | AT kaiguo multimodalmedicalimagefusionbasedonfusionnetinyiqcolorspace AT xiongfeili multimodalmedicalimagefusionbasedonfusionnetinyiqcolorspace AT hongruizang multimodalmedicalimagefusionbasedonfusionnetinyiqcolorspace AT tiehufan multimodalmedicalimagefusionbasedonfusionnetinyiqcolorspace |