Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer
Medical image segmentation is a vital healthcare endeavor requiring precise and efficient models for appropriate diagnosis and treatment. Vision transformer (ViT)-based segmentation models have shown great performance in accomplishing this task. However, to build a powerful backbone, the self-attent...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/10/7/879 |
| _version_ | 1797590231107502080 |
|---|---|
| author | Numan Saeed Muhammad Ridzuan Roba Al Majzoub Mohammad Yaqub |
| author_facet | Numan Saeed Muhammad Ridzuan Roba Al Majzoub Mohammad Yaqub |
| author_sort | Numan Saeed |
| collection | DOAJ |
| description | Medical image segmentation is a vital healthcare endeavor requiring precise and efficient models for appropriate diagnosis and treatment. Vision transformer (ViT)-based segmentation models have shown great performance in accomplishing this task. However, to build a powerful backbone, the self-attention block of ViT requires large-scale pre-training data. The present method of modifying pre-trained models entails updating all or some of the backbone parameters. This paper proposes a novel fine-tuning strategy for adapting a pretrained transformer-based segmentation model on data from a new medical center. This method introduces a small number of learnable parameters, termed prompts, into the input space (less than 1% of model parameters) while keeping the rest of the model parameters frozen. Extensive studies employing data from new unseen medical centers show that the prompt-based fine-tuning of medical segmentation models provides excellent performance regarding the new-center data with a negligible drop regarding the old centers. Additionally, our strategy delivers great accuracy with minimum re-training on new-center data, significantly decreasing the computational and time costs of fine-tuning pre-trained models. Our source code will be made publicly available. |
| first_indexed | 2024-03-11T01:17:36Z |
| format | Article |
| id | doaj.art-6838b7d35aa9437ca6f6cce387b20e65 |
| institution | Directory Open Access Journal |
| issn | 2306-5354 |
| language | English |
| last_indexed | 2024-03-11T01:17:36Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj.art-6838b7d35aa9437ca6f6cce387b20e652023-11-18T18:22:38ZengMDPI AGBioengineering2306-53542023-07-0110787910.3390/bioengineering10070879Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck CancerNuman Saeed0Muhammad Ridzuan1Roba Al Majzoub2Mohammad Yaqub3Department of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi 7909, United Arab EmiratesDepartment of Machine Learning, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi 7909, United Arab EmiratesDepartment of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi 7909, United Arab EmiratesDepartment of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi 7909, United Arab EmiratesMedical image segmentation is a vital healthcare endeavor requiring precise and efficient models for appropriate diagnosis and treatment. Vision transformer (ViT)-based segmentation models have shown great performance in accomplishing this task. However, to build a powerful backbone, the self-attention block of ViT requires large-scale pre-training data. The present method of modifying pre-trained models entails updating all or some of the backbone parameters. This paper proposes a novel fine-tuning strategy for adapting a pretrained transformer-based segmentation model on data from a new medical center. This method introduces a small number of learnable parameters, termed prompts, into the input space (less than 1% of model parameters) while keeping the rest of the model parameters frozen. Extensive studies employing data from new unseen medical centers show that the prompt-based fine-tuning of medical segmentation models provides excellent performance regarding the new-center data with a negligible drop regarding the old centers. Additionally, our strategy delivers great accuracy with minimum re-training on new-center data, significantly decreasing the computational and time costs of fine-tuning pre-trained models. Our source code will be made publicly available.https://www.mdpi.com/2306-5354/10/7/879transfer learningvision transformermedical image segmentationlimited dataprompt-based tuning |
| spellingShingle | Numan Saeed Muhammad Ridzuan Roba Al Majzoub Mohammad Yaqub Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer Bioengineering transfer learning vision transformer medical image segmentation limited data prompt-based tuning |
| title | Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer |
| title_full | Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer |
| title_fullStr | Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer |
| title_full_unstemmed | Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer |
| title_short | Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer |
| title_sort | prompt based tuning of transformer models for multi center medical image segmentation of head and neck cancer |
| topic | transfer learning vision transformer medical image segmentation limited data prompt-based tuning |
| url | https://www.mdpi.com/2306-5354/10/7/879 |
| work_keys_str_mv | AT numansaeed promptbasedtuningoftransformermodelsformulticentermedicalimagesegmentationofheadandneckcancer AT muhammadridzuan promptbasedtuningoftransformermodelsformulticentermedicalimagesegmentationofheadandneckcancer AT robaalmajzoub promptbasedtuningoftransformermodelsformulticentermedicalimagesegmentationofheadandneckcancer AT mohammadyaqub promptbasedtuningoftransformermodelsformulticentermedicalimagesegmentationofheadandneckcancer |