S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications
With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-l...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-11-01
|
| Series: | Frontiers in Physiology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fphys.2023.1209659/full |
| _version_ | 1797640156141846528 |
|---|---|
| author | Nan Mu Nan Mu Zonghan Lyu Zonghan Lyu Mostafa Rezaeitaleshmahalleh Mostafa Rezaeitaleshmahalleh Cassie Bonifas Cassie Bonifas Jordan Gosnell Marcus Haw Joseph Vettukattil Joseph Vettukattil Jingfeng Jiang Jingfeng Jiang |
| author_facet | Nan Mu Nan Mu Zonghan Lyu Zonghan Lyu Mostafa Rezaeitaleshmahalleh Mostafa Rezaeitaleshmahalleh Cassie Bonifas Cassie Bonifas Jordan Gosnell Marcus Haw Joseph Vettukattil Joseph Vettukattil Jingfeng Jiang Jingfeng Jiang |
| author_sort | Nan Mu |
| collection | DOAJ |
| description | With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images. |
| first_indexed | 2024-03-11T13:26:51Z |
| format | Article |
| id | doaj.art-cc4892c94c644972a65e0810bdcb5573 |
| institution | Directory Open Access Journal |
| issn | 1664-042X |
| language | English |
| last_indexed | 2024-03-11T13:26:51Z |
| publishDate | 2023-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Physiology |
| spelling | doaj.art-cc4892c94c644972a65e0810bdcb55732023-11-03T06:08:44ZengFrontiers Media S.A.Frontiers in Physiology1664-042X2023-11-011410.3389/fphys.2023.12096591209659S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applicationsNan Mu0Nan Mu1Zonghan Lyu2Zonghan Lyu3Mostafa Rezaeitaleshmahalleh4Mostafa Rezaeitaleshmahalleh5Cassie Bonifas6Cassie Bonifas7Jordan Gosnell8Marcus Haw9Joseph Vettukattil10Joseph Vettukattil11Jingfeng Jiang12Jingfeng Jiang13Department of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesCenter for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United StatesDepartment of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesCenter for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United StatesDepartment of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesCenter for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United StatesDepartment of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesCenter for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United StatesBetz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United StatesBetz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United StatesDepartment of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesBetz Congenital Health Center, Helen DeVos Children’s Hospital, Grand Rapids, MI, United StatesDepartment of Biomedical Engineering, Michigan Technological University, Houghton, MI, United StatesCenter for Biocomputing and Digital Health, Health Research Institute, Institute of Computing and Cybernetics, Michigan Technological University, Houghton, MI, United StatesWith the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images.https://www.frontiersin.org/articles/10.3389/fphys.2023.1209659/fullautomatic segmentationcardiac wallintracranial aneurysm (IA)small/thin structurefully convolutional network (FCN) |
| spellingShingle | Nan Mu Nan Mu Zonghan Lyu Zonghan Lyu Mostafa Rezaeitaleshmahalleh Mostafa Rezaeitaleshmahalleh Cassie Bonifas Cassie Bonifas Jordan Gosnell Marcus Haw Joseph Vettukattil Joseph Vettukattil Jingfeng Jiang Jingfeng Jiang S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications Frontiers in Physiology automatic segmentation cardiac wall intracranial aneurysm (IA) small/thin structure fully convolutional network (FCN) |
| title | S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications |
| title_full | S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications |
| title_fullStr | S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications |
| title_full_unstemmed | S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications |
| title_short | S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications |
| title_sort | s net a multiple cross aggregation convolutional architecture for automatic segmentation of small thin structures for cardiovascular applications |
| topic | automatic segmentation cardiac wall intracranial aneurysm (IA) small/thin structure fully convolutional network (FCN) |
| url | https://www.frontiersin.org/articles/10.3389/fphys.2023.1209659/full |
| work_keys_str_mv | AT nanmu snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT nanmu snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT zonghanlyu snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT zonghanlyu snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT mostafarezaeitaleshmahalleh snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT mostafarezaeitaleshmahalleh snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT cassiebonifas snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT cassiebonifas snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT jordangosnell snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT marcushaw snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT josephvettukattil snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT josephvettukattil snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT jingfengjiang snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications AT jingfengjiang snetamultiplecrossaggregationconvolutionalarchitectureforautomaticsegmentationofsmallthinstructuresforcardiovascularapplications |