ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation
Convolutional Neural Network (CNN) plays a vital role in the development of computer vision applications. The depth neural network composed of U-shaped structures and jump connections is widely used in various medical image tasks. Recently, based on the self-attention mechanism, the Transformer stru...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2023-01-01
|
| Series: | Mathematical Biosciences and Engineering |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/mbe.2023007?viewType=HTML |
| _version_ | 1797990644496465920 |
|---|---|
| author | Xiaomeng Feng Taiping Wang Xiaohang Yang Minfei Zhang Wanpeng Guo Weina Wang |
| author_facet | Xiaomeng Feng Taiping Wang Xiaohang Yang Minfei Zhang Wanpeng Guo Weina Wang |
| author_sort | Xiaomeng Feng |
| collection | DOAJ |
| description | Convolutional Neural Network (CNN) plays a vital role in the development of computer vision applications. The depth neural network composed of U-shaped structures and jump connections is widely used in various medical image tasks. Recently, based on the self-attention mechanism, the Transformer structure has made great progress and tends to replace CNN, and it has great advantages in understanding global information. In this paper, the ConvWin Transformer structure is proposed, which refers to the W-MSA structure in Swin and combines with the convolution. It can not only accelerate the convergence speed, but also enrich the information exchange between patches and improve the understanding of local information. Then, it is integrated with UNet, a U-shaped architecture commonly used in medical image segmentation, to form a structure called ConvWin-UNet. Meanwhile, this paper improves the patch expanding layer to perform the upsampling operation. The experimental results on the Hubmap datasets and synapse multi-organ segmentation dataset indicate that the proposed ConvWin-UNet structure achieves excellent results. Partial code and models of this work are available at https://github.com/xmFeng-hdu/ConvWin-UNet. |
| first_indexed | 2024-04-11T08:39:45Z |
| format | Article |
| id | doaj.art-5bd8bad4a3d14e30bc4c31bca4b7f23c |
| institution | Directory Open Access Journal |
| issn | 1551-0018 |
| language | English |
| last_indexed | 2024-04-11T08:39:45Z |
| publishDate | 2023-01-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | Mathematical Biosciences and Engineering |
| spelling | doaj.art-5bd8bad4a3d14e30bc4c31bca4b7f23c2022-12-22T04:34:14ZengAIMS PressMathematical Biosciences and Engineering1551-00182023-01-0120112814410.3934/mbe.2023007ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentationXiaomeng Feng0Taiping Wang1Xiaohang Yang2Minfei Zhang3Wanpeng Guo4Weina Wang 51. Department of Mathematics, School of Sciences, Hangzhou Dianzi University, Hangzhou 310018, China2. Hangzhou Medipath Intelligent Technology Co., Ltd, Hangzhou, China 3. School of Business, Macau University of Science and Technology, Macau, China2. Hangzhou Medipath Intelligent Technology Co., Ltd, Hangzhou, China2. Hangzhou Medipath Intelligent Technology Co., Ltd, Hangzhou, China2. Hangzhou Medipath Intelligent Technology Co., Ltd, Hangzhou, China1. Department of Mathematics, School of Sciences, Hangzhou Dianzi University, Hangzhou 310018, ChinaConvolutional Neural Network (CNN) plays a vital role in the development of computer vision applications. The depth neural network composed of U-shaped structures and jump connections is widely used in various medical image tasks. Recently, based on the self-attention mechanism, the Transformer structure has made great progress and tends to replace CNN, and it has great advantages in understanding global information. In this paper, the ConvWin Transformer structure is proposed, which refers to the W-MSA structure in Swin and combines with the convolution. It can not only accelerate the convergence speed, but also enrich the information exchange between patches and improve the understanding of local information. Then, it is integrated with UNet, a U-shaped architecture commonly used in medical image segmentation, to form a structure called ConvWin-UNet. Meanwhile, this paper improves the patch expanding layer to perform the upsampling operation. The experimental results on the Hubmap datasets and synapse multi-organ segmentation dataset indicate that the proposed ConvWin-UNet structure achieves excellent results. Partial code and models of this work are available at https://github.com/xmFeng-hdu/ConvWin-UNet.https://www.aimspress.com/article/doi/10.3934/mbe.2023007?viewType=HTMLconvwin-unetvision transformerconvolutionunetsegmentation |
| spellingShingle | Xiaomeng Feng Taiping Wang Xiaohang Yang Minfei Zhang Wanpeng Guo Weina Wang ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation Mathematical Biosciences and Engineering convwin-unet vision transformer convolution unet segmentation |
| title | ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation |
| title_full | ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation |
| title_fullStr | ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation |
| title_full_unstemmed | ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation |
| title_short | ConvWin-UNet: UNet-like hierarchical vision Transformer combined with convolution for medical image segmentation |
| title_sort | convwin unet unet like hierarchical vision transformer combined with convolution for medical image segmentation |
| topic | convwin-unet vision transformer convolution unet segmentation |
| url | https://www.aimspress.com/article/doi/10.3934/mbe.2023007?viewType=HTML |
| work_keys_str_mv | AT xiaomengfeng convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation AT taipingwang convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation AT xiaohangyang convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation AT minfeizhang convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation AT wanpengguo convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation AT weinawang convwinunetunetlikehierarchicalvisiontransformercombinedwithconvolutionformedicalimagesegmentation |