STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation
Acquisition of high quality manual annotations is vital for the development of segmentation algorithms. However, to create them we require a substantial amount of expert time and knowledge. Large numbers of labels are required to train convolutional neural networks due to the vast number of paramete...
| Main Authors: | , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Elsevier
2022
|
| _version_ | 1797107870497505280 |
|---|---|
| author | Dinsdale, NK Jenkinson, M Namburete, AIL |
| author_facet | Dinsdale, NK Jenkinson, M Namburete, AIL |
| author_sort | Dinsdale, NK |
| collection | OXFORD |
| description | Acquisition of high quality manual annotations is vital for the development of segmentation algorithms. However, to create them we require a substantial amount of expert time and knowledge. Large numbers of labels are required to train convolutional neural networks due to the vast number of parameters that must be learned in the optimisation process. Here, we develop the STAMP algorithm to allow the simultaneous training and pruning of a UNet architecture for medical image segmentation with targeted channelwise dropout to make the network robust to the pruning. We demonstrate the technique across segmentation tasks and imaging modalities. It is then shown that, through online pruning, we are able to train networks to have much higher performance than the equivalent standard UNet models while reducing their size by more than 85% in terms of parameters. This has the potential to allow networks to be directly trained on datasets where very low numbers of labels are available. |
| first_indexed | 2024-03-07T07:21:44Z |
| format | Journal article |
| id | oxford-uuid:b8590014-471e-4876-baf4-00b6b74bb8c5 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:21:44Z |
| publishDate | 2022 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | oxford-uuid:b8590014-471e-4876-baf4-00b6b74bb8c52022-10-14T16:07:21ZSTAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentationJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:b8590014-471e-4876-baf4-00b6b74bb8c5EnglishSymplectic ElementsElsevier2022Dinsdale, NKJenkinson, MNamburete, AILAcquisition of high quality manual annotations is vital for the development of segmentation algorithms. However, to create them we require a substantial amount of expert time and knowledge. Large numbers of labels are required to train convolutional neural networks due to the vast number of parameters that must be learned in the optimisation process. Here, we develop the STAMP algorithm to allow the simultaneous training and pruning of a UNet architecture for medical image segmentation with targeted channelwise dropout to make the network robust to the pruning. We demonstrate the technique across segmentation tasks and imaging modalities. It is then shown that, through online pruning, we are able to train networks to have much higher performance than the equivalent standard UNet models while reducing their size by more than 85% in terms of parameters. This has the potential to allow networks to be directly trained on datasets where very low numbers of labels are available. |
| spellingShingle | Dinsdale, NK Jenkinson, M Namburete, AIL STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title | STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title_full | STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title_fullStr | STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title_full_unstemmed | STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title_short | STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation |
| title_sort | stamp simultaneous training and model pruning for low data regimes in medical image segmentation |
| work_keys_str_mv | AT dinsdalenk stampsimultaneoustrainingandmodelpruningforlowdataregimesinmedicalimagesegmentation AT jenkinsonm stampsimultaneoustrainingandmodelpruningforlowdataregimesinmedicalimagesegmentation AT nambureteail stampsimultaneoustrainingandmodelpruningforlowdataregimesinmedicalimagesegmentation |