FFUNet: A novel feature fusion makes strong decoder for medical image segmentation
Abstract Convolutional neural networks (CNNs) have strong ability to extract local features, but it is slightly lacking in extracting global contexts. In contrast, transformers are good at long‐distance modelling due to the global self‐attention mechanisms while its performance in localization is li...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-07-01
|
| Series: | IET Signal Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/sil2.12114 |
| _version_ | 1826820755286917120 |
|---|---|
| author | Junsong Xie Renju Zhu Zezhi Wu Jinling Ouyang |
| author_facet | Junsong Xie Renju Zhu Zezhi Wu Jinling Ouyang |
| author_sort | Junsong Xie |
| collection | DOAJ |
| description | Abstract Convolutional neural networks (CNNs) have strong ability to extract local features, but it is slightly lacking in extracting global contexts. In contrast, transformers are good at long‐distance modelling due to the global self‐attention mechanisms while its performance in localization is limited. On the other hand, the feature gap between an encoder and decoder is also challenging for a U‐shaped network, which adopts a plain skip connection. Inherited from convolutional networks and transformers, FFUNet, a hybrid network structure with a novel module named the Feature Fusion Module (FFM) is proposed for medical image segmentation. The proposed FFM consisting of Feature Attention Selection, Cross Offset Generation and Deformable Convolution Layer, aims to replace the original plain skip connection to alleviate the ambiguous semantic information between the encoder and decoder for a more powerful medical image segmentation network. Extensive experiments demonstrate that the proposed FFUNet has amazing performance in segmentation gains on the Synapse dataset. In addition, consistent improvements are also achieved across other four popular datasets and CNN‐based or transformer‐based segmentation networks, which illustrate that the proposed method has advantages in generalization and compactness. |
| first_indexed | 2024-03-09T09:09:57Z |
| format | Article |
| id | doaj.art-715d65db62a94777bdec2d0b544b9cda |
| institution | Directory Open Access Journal |
| issn | 1751-9675 1751-9683 |
| language | English |
| last_indexed | 2025-02-16T06:34:52Z |
| publishDate | 2022-07-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Signal Processing |
| spelling | doaj.art-715d65db62a94777bdec2d0b544b9cda2025-02-03T06:45:05ZengWileyIET Signal Processing1751-96751751-96832022-07-0116550151410.1049/sil2.12114FFUNet: A novel feature fusion makes strong decoder for medical image segmentationJunsong Xie0Renju Zhu1Zezhi Wu2Jinling Ouyang3Department of Computer Science Anhui Medical University Hefei ChinaDepartment of Critical Care Medicine Hefei BOE Hospital Hefei ChinaDepartment of Computer Science Anhui Medical University Hefei ChinaFirst Clinical Medical College Anhui Medical University Hefei ChinaAbstract Convolutional neural networks (CNNs) have strong ability to extract local features, but it is slightly lacking in extracting global contexts. In contrast, transformers are good at long‐distance modelling due to the global self‐attention mechanisms while its performance in localization is limited. On the other hand, the feature gap between an encoder and decoder is also challenging for a U‐shaped network, which adopts a plain skip connection. Inherited from convolutional networks and transformers, FFUNet, a hybrid network structure with a novel module named the Feature Fusion Module (FFM) is proposed for medical image segmentation. The proposed FFM consisting of Feature Attention Selection, Cross Offset Generation and Deformable Convolution Layer, aims to replace the original plain skip connection to alleviate the ambiguous semantic information between the encoder and decoder for a more powerful medical image segmentation network. Extensive experiments demonstrate that the proposed FFUNet has amazing performance in segmentation gains on the Synapse dataset. In addition, consistent improvements are also achieved across other four popular datasets and CNN‐based or transformer‐based segmentation networks, which illustrate that the proposed method has advantages in generalization and compactness.https://doi.org/10.1049/sil2.12114attention mechanismconformerdeformable convolutionfeature fusionmedical image segmentationskip connection |
| spellingShingle | Junsong Xie Renju Zhu Zezhi Wu Jinling Ouyang FFUNet: A novel feature fusion makes strong decoder for medical image segmentation IET Signal Processing attention mechanism conformer deformable convolution feature fusion medical image segmentation skip connection |
| title | FFUNet: A novel feature fusion makes strong decoder for medical image segmentation |
| title_full | FFUNet: A novel feature fusion makes strong decoder for medical image segmentation |
| title_fullStr | FFUNet: A novel feature fusion makes strong decoder for medical image segmentation |
| title_full_unstemmed | FFUNet: A novel feature fusion makes strong decoder for medical image segmentation |
| title_short | FFUNet: A novel feature fusion makes strong decoder for medical image segmentation |
| title_sort | ffunet a novel feature fusion makes strong decoder for medical image segmentation |
| topic | attention mechanism conformer deformable convolution feature fusion medical image segmentation skip connection |
| url | https://doi.org/10.1049/sil2.12114 |
| work_keys_str_mv | AT junsongxie ffunetanovelfeaturefusionmakesstrongdecoderformedicalimagesegmentation AT renjuzhu ffunetanovelfeaturefusionmakesstrongdecoderformedicalimagesegmentation AT zezhiwu ffunetanovelfeaturefusionmakesstrongdecoderformedicalimagesegmentation AT jinlingouyang ffunetanovelfeaturefusionmakesstrongdecoderformedicalimagesegmentation |