Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation
This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism (PE) lesion areas in computed tomography pulmonary angiogram (CTPA) images. In the current study, all of the PE CTPA image segmentation methods were trained...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-05-01
|
| Series: | Heliyon |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S240584402303267X |
| _version_ | 1797815550184783872 |
|---|---|
| author | Ting-Wei Cheng Yi Wei Chua Ching-Chun Huang Jerry Chang Chin Kuo Yun-Chien Cheng |
| author_facet | Ting-Wei Cheng Yi Wei Chua Ching-Chun Huang Jerry Chang Chin Kuo Yun-Chien Cheng |
| author_sort | Ting-Wei Cheng |
| collection | DOAJ |
| description | This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism (PE) lesion areas in computed tomography pulmonary angiogram (CTPA) images. In the current study, all of the PE CTPA image segmentation methods were trained by supervised learning. However, when CTPA images come from different hospitals, the supervised learning models need to be retrained and the images need to be relabeled. Therefore, this study proposed a semi-supervised learning method to make the model applicable to different datasets by the addition of a small number of unlabeled images. By training the model with both labeled and unlabeled images, the accuracy of unlabeled images was improved and the labeling cost was reduced. Our proposed semi-supervised segmentation model included a segmentation network and a discriminator network. We added feature information generated from the encoder of the segmentation network to the discriminator so that it could learn the similarities between the prediction label and ground truth label. The HRNet-based architecture was modified and used as the segmentation network. This HRNet-based architecture could maintain a higher resolution for convolutional operations to improve the prediction of small PE lesion areas. We used a labeled open-source dataset and an unlabeled National Cheng Kung University Hospital (NCKUH) (IRB number: B-ER-108-380) dataset to train the semi-supervised learning model, and the resulting mean intersection over union (mIOU), dice score, and sensitivity reached 0.3510, 0.4854, and 0.4253, respectively, on the NCKUH dataset. Then we fine-tuned and tested the model with a small number of unlabeled PE CTPA images in a dataset from China Medical University Hospital (CMUH) (IRB number: CMUH110-REC3-173). Comparing the results of our semi-supervised model with those of the supervised model, the mIOU, dice score, and sensitivity improved from 0.2344, 0.3325, and 0.3151 to 0.3721, 0.5113, and 0.4967, respectively. In conclusion, our semi-supervised model can improve the accuracy on other datasets and reduce the labor cost of labeling with the use of only a small number of unlabeled images for fine-tuning. |
| first_indexed | 2024-03-13T08:24:30Z |
| format | Article |
| id | doaj.art-140c2fd6b9284ce0bb51a61d85da16f3 |
| institution | Directory Open Access Journal |
| issn | 2405-8440 |
| language | English |
| last_indexed | 2024-03-13T08:24:30Z |
| publishDate | 2023-05-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Heliyon |
| spelling | doaj.art-140c2fd6b9284ce0bb51a61d85da16f32023-05-31T04:46:28ZengElsevierHeliyon2405-84402023-05-0195e16060Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotationTing-Wei Cheng0Yi Wei Chua1Ching-Chun Huang2Jerry Chang3Chin Kuo4Yun-Chien Cheng5Department of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University, Hsin-Chu, TaiwanDepartment of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University, Hsin-Chu, TaiwanDepartment of Computer Science, College of Computer Science, National Yang Ming Chiao Tung University, Hsin-Chu, TaiwanDepartment of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University, Hsin-Chu, TaiwanDepartment of Oncology, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan; College of Artificial Intelligence, National Yang Ming Chiao Tung University, Hsin-Chu, Taiwan; Corresponding author. Department of Oncology, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan.Department of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University, Hsin-Chu, Taiwan; Corresponding author. Department of Mechanical Engineering, College of Engineering, National Yang Ming Chiao Tung University, Hsin-Chu, Taiwan.This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism (PE) lesion areas in computed tomography pulmonary angiogram (CTPA) images. In the current study, all of the PE CTPA image segmentation methods were trained by supervised learning. However, when CTPA images come from different hospitals, the supervised learning models need to be retrained and the images need to be relabeled. Therefore, this study proposed a semi-supervised learning method to make the model applicable to different datasets by the addition of a small number of unlabeled images. By training the model with both labeled and unlabeled images, the accuracy of unlabeled images was improved and the labeling cost was reduced. Our proposed semi-supervised segmentation model included a segmentation network and a discriminator network. We added feature information generated from the encoder of the segmentation network to the discriminator so that it could learn the similarities between the prediction label and ground truth label. The HRNet-based architecture was modified and used as the segmentation network. This HRNet-based architecture could maintain a higher resolution for convolutional operations to improve the prediction of small PE lesion areas. We used a labeled open-source dataset and an unlabeled National Cheng Kung University Hospital (NCKUH) (IRB number: B-ER-108-380) dataset to train the semi-supervised learning model, and the resulting mean intersection over union (mIOU), dice score, and sensitivity reached 0.3510, 0.4854, and 0.4253, respectively, on the NCKUH dataset. Then we fine-tuned and tested the model with a small number of unlabeled PE CTPA images in a dataset from China Medical University Hospital (CMUH) (IRB number: CMUH110-REC3-173). Comparing the results of our semi-supervised model with those of the supervised model, the mIOU, dice score, and sensitivity improved from 0.2344, 0.3325, and 0.3151 to 0.3721, 0.5113, and 0.4967, respectively. In conclusion, our semi-supervised model can improve the accuracy on other datasets and reduce the labor cost of labeling with the use of only a small number of unlabeled images for fine-tuning.http://www.sciencedirect.com/science/article/pii/S240584402303267XPulmonary embolismComputed tomography pulmonary angiogramSemantic segmentationSemi-supervised learningUnlabeled images |
| spellingShingle | Ting-Wei Cheng Yi Wei Chua Ching-Chun Huang Jerry Chang Chin Kuo Yun-Chien Cheng Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation Heliyon Pulmonary embolism Computed tomography pulmonary angiogram Semantic segmentation Semi-supervised learning Unlabeled images |
| title | Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation |
| title_full | Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation |
| title_fullStr | Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation |
| title_full_unstemmed | Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation |
| title_short | Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation |
| title_sort | feature enhanced adversarial semi supervised semantic segmentation network for pulmonary embolism annotation |
| topic | Pulmonary embolism Computed tomography pulmonary angiogram Semantic segmentation Semi-supervised learning Unlabeled images |
| url | http://www.sciencedirect.com/science/article/pii/S240584402303267X |
| work_keys_str_mv | AT tingweicheng featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation AT yiweichua featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation AT chingchunhuang featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation AT jerrychang featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation AT chinkuo featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation AT yunchiencheng featureenhancedadversarialsemisupervisedsemanticsegmentationnetworkforpulmonaryembolismannotation |