Self-supervised multi-modal alignment for whole body medical imaging
This paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic res...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2021
|
| _version_ | 1826276194043035648 |
|---|---|
| author | Windsor, R Jamaludin, A Kadir, T Zisserman, A |
| author2 | de Bruijne, M |
| author_facet | de Bruijne, M Windsor, R Jamaludin, A Kadir, T Zisserman, A |
| author_sort | Windsor, R |
| collection | OXFORD |
| description | This paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans. We make three contributions: (i) We introduce a multi-modal image-matching contrastive framework, that is able to learn to match different-modality scans of the same subject with high accuracy. (ii) Without any adaption, we show that the correspondences learnt during this contrastive training step can be used to perform automatic cross-modal scan registration in a completely unsupervised manner. (iii) Finally, we use these registrations to transfer segmentation maps from the DXA scans to the MR scans where they are used to train a network to segment anatomical regions without requiring ground-truth MR examples. To aid further research, our code is publicly available (https://github.com/rwindsor1/biobank-self-supervised-alignment). |
| first_indexed | 2024-03-06T23:10:20Z |
| format | Conference item |
| id | oxford-uuid:653f5a49-2c55-43d4-a8fe-57b6382f7748 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T23:10:20Z |
| publishDate | 2021 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:653f5a49-2c55-43d4-a8fe-57b6382f77482022-03-26T18:24:19ZSelf-supervised multi-modal alignment for whole body medical imagingConference itemhttp://purl.org/coar/resource_type/c_5794uuid:653f5a49-2c55-43d4-a8fe-57b6382f7748EnglishSymplectic ElementsSpringer2021Windsor, RJamaludin, AKadir, TZisserman, Ade Bruijne, MCattin, PCCotin, SPadoy, NSpeidel, SZheng, YEssert, CThis paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans. We make three contributions: (i) We introduce a multi-modal image-matching contrastive framework, that is able to learn to match different-modality scans of the same subject with high accuracy. (ii) Without any adaption, we show that the correspondences learnt during this contrastive training step can be used to perform automatic cross-modal scan registration in a completely unsupervised manner. (iii) Finally, we use these registrations to transfer segmentation maps from the DXA scans to the MR scans where they are used to train a network to segment anatomical regions without requiring ground-truth MR examples. To aid further research, our code is publicly available (https://github.com/rwindsor1/biobank-self-supervised-alignment). |
| spellingShingle | Windsor, R Jamaludin, A Kadir, T Zisserman, A Self-supervised multi-modal alignment for whole body medical imaging |
| title | Self-supervised multi-modal alignment for whole body medical imaging |
| title_full | Self-supervised multi-modal alignment for whole body medical imaging |
| title_fullStr | Self-supervised multi-modal alignment for whole body medical imaging |
| title_full_unstemmed | Self-supervised multi-modal alignment for whole body medical imaging |
| title_short | Self-supervised multi-modal alignment for whole body medical imaging |
| title_sort | self supervised multi modal alignment for whole body medical imaging |
| work_keys_str_mv | AT windsorr selfsupervisedmultimodalalignmentforwholebodymedicalimaging AT jamaludina selfsupervisedmultimodalalignmentforwholebodymedicalimaging AT kadirt selfsupervisedmultimodalalignmentforwholebodymedicalimaging AT zissermana selfsupervisedmultimodalalignmentforwholebodymedicalimaging |