HUT: Hybrid UNet transformer for brain lesion and tumour segmentation
A supervised deep learning network like the UNet has performed well in segmenting brain anomalies such as lesions and tumours. However, such methods were proposed to perform on single-modality or multi-modality images. We use the Hybrid UNet Transformer (HUT) to improve performance in single-modalit...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-12-01
|
| Series: | Heliyon |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2405844023096202 |
| _version_ | 1797384027486814208 |
|---|---|
| author | Wei Kwek Soh Hing Yee Yuen Jagath C. Rajapakse |
| author_facet | Wei Kwek Soh Hing Yee Yuen Jagath C. Rajapakse |
| author_sort | Wei Kwek Soh |
| collection | DOAJ |
| description | A supervised deep learning network like the UNet has performed well in segmenting brain anomalies such as lesions and tumours. However, such methods were proposed to perform on single-modality or multi-modality images. We use the Hybrid UNet Transformer (HUT) to improve performance in single-modality lesion segmentation and multi-modality brain tumour segmentation. The HUT consists of two pipelines running in parallel, one of which is UNet-based and the other is Transformer-based. The Transformer-based pipeline relies on feature maps in the intermediate layers of the UNet decoder during training. The HUT network takes in the available modalities of 3D brain volumes and embeds the brain volumes into voxel patches. The transformers in the system improve global attention and long-range correlation between the voxel patches. In addition, we introduce a self-supervised training approach in the HUT framework to enhance the overall segmentation performance. We demonstrate that HUT performs better than the state-of-the-art network SPiN in the single-modality segmentation on Anatomical Tracings of Lesions After Stroke (ATLAS) dataset by 4.84% of Dice score and a significant 41% in the Hausdorff Distance score. HUT also performed well on brain scans in the Brain Tumour Segmentation (BraTS20) dataset and demonstrated an improvement over the state-of-the-art network nnUnet by 0.96% in the Dice score and 4.1% in the Hausdorff Distance score. |
| first_indexed | 2024-03-08T21:29:29Z |
| format | Article |
| id | doaj.art-c46d12ec6fdd4abcb154e939cbdbfeed |
| institution | Directory Open Access Journal |
| issn | 2405-8440 |
| language | English |
| last_indexed | 2024-03-08T21:29:29Z |
| publishDate | 2023-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Heliyon |
| spelling | doaj.art-c46d12ec6fdd4abcb154e939cbdbfeed2023-12-21T07:33:29ZengElsevierHeliyon2405-84402023-12-01912e22412HUT: Hybrid UNet transformer for brain lesion and tumour segmentationWei Kwek Soh0Hing Yee Yuen1Jagath C. Rajapakse2Nanyang Technological University Biomedical Informatics Lab, Block NS4-04-33 50 Nanyang Avenue, Singapore, 639798, Singapore, SingaporeNanyang Technological University Biomedical Informatics Lab, Block NS4-04-33 50 Nanyang Avenue, Singapore, 639798, Singapore, SingaporeCorresponding author.; Nanyang Technological University Biomedical Informatics Lab, Block NS4-04-33 50 Nanyang Avenue, Singapore, 639798, Singapore, SingaporeA supervised deep learning network like the UNet has performed well in segmenting brain anomalies such as lesions and tumours. However, such methods were proposed to perform on single-modality or multi-modality images. We use the Hybrid UNet Transformer (HUT) to improve performance in single-modality lesion segmentation and multi-modality brain tumour segmentation. The HUT consists of two pipelines running in parallel, one of which is UNet-based and the other is Transformer-based. The Transformer-based pipeline relies on feature maps in the intermediate layers of the UNet decoder during training. The HUT network takes in the available modalities of 3D brain volumes and embeds the brain volumes into voxel patches. The transformers in the system improve global attention and long-range correlation between the voxel patches. In addition, we introduce a self-supervised training approach in the HUT framework to enhance the overall segmentation performance. We demonstrate that HUT performs better than the state-of-the-art network SPiN in the single-modality segmentation on Anatomical Tracings of Lesions After Stroke (ATLAS) dataset by 4.84% of Dice score and a significant 41% in the Hausdorff Distance score. HUT also performed well on brain scans in the Brain Tumour Segmentation (BraTS20) dataset and demonstrated an improvement over the state-of-the-art network nnUnet by 0.96% in the Dice score and 4.1% in the Hausdorff Distance score.http://www.sciencedirect.com/science/article/pii/S2405844023096202Brain tumourBrain lesionsMultimodal MRISingle-modal MRISelf-supervised segmentationVision transformer |
| spellingShingle | Wei Kwek Soh Hing Yee Yuen Jagath C. Rajapakse HUT: Hybrid UNet transformer for brain lesion and tumour segmentation Heliyon Brain tumour Brain lesions Multimodal MRI Single-modal MRI Self-supervised segmentation Vision transformer |
| title | HUT: Hybrid UNet transformer for brain lesion and tumour segmentation |
| title_full | HUT: Hybrid UNet transformer for brain lesion and tumour segmentation |
| title_fullStr | HUT: Hybrid UNet transformer for brain lesion and tumour segmentation |
| title_full_unstemmed | HUT: Hybrid UNet transformer for brain lesion and tumour segmentation |
| title_short | HUT: Hybrid UNet transformer for brain lesion and tumour segmentation |
| title_sort | hut hybrid unet transformer for brain lesion and tumour segmentation |
| topic | Brain tumour Brain lesions Multimodal MRI Single-modal MRI Self-supervised segmentation Vision transformer |
| url | http://www.sciencedirect.com/science/article/pii/S2405844023096202 |
| work_keys_str_mv | AT weikweksoh huthybridunettransformerforbrainlesionandtumoursegmentation AT hingyeeyuen huthybridunettransformerforbrainlesionandtumoursegmentation AT jagathcrajapakse huthybridunettransformerforbrainlesionandtumoursegmentation |