Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in thi...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2019
|
| _version_ | 1797084785813749760 |
|---|---|
| author | Jiao, J Namburete, AIL Papageorghiou, AT Noble, JA |
| author_facet | Jiao, J Namburete, AIL Papageorghiou, AT Noble, JA |
| author_sort | Jiao, J |
| collection | OXFORD |
| description | Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for instance, for US-MRI registration or fusion. Our model is self-supervised and end-to-end trainable. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise an extractor to determine shared latent features, which are then used for data synthesis. Since paired data was unavailable for our study (and rare in practice), we propose to enforce the distributions to be similar instead of employing pixel-wise constraints, by adversarial learning in both the image domain and latent space. Furthermore, we propose an adversarial structural constraint to regularise the anatomical structures between the two modalities during the synthesis. A cross-modal attention scheme is proposed to leverage non-local spatial correlations. The feasibility of the approach to produce realistic looking MR images is demonstrated quantitatively and with a qualitative evaluation compared to real fetal MR images. |
| first_indexed | 2024-03-07T01:59:55Z |
| format | Conference item |
| id | oxford-uuid:9cfd4d89-25c7-49f9-b977-be0694c96fa6 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T01:59:55Z |
| publishDate | 2019 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:9cfd4d89-25c7-49f9-b977-be0694c96fa62022-03-27T00:39:56ZAnatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound imagesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:9cfd4d89-25c7-49f9-b977-be0694c96fa6EnglishSymplectic ElementsSpringer2019Jiao, JNamburete, AILPapageorghiou, ATNoble, JAFetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for instance, for US-MRI registration or fusion. Our model is self-supervised and end-to-end trainable. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise an extractor to determine shared latent features, which are then used for data synthesis. Since paired data was unavailable for our study (and rare in practice), we propose to enforce the distributions to be similar instead of employing pixel-wise constraints, by adversarial learning in both the image domain and latent space. Furthermore, we propose an adversarial structural constraint to regularise the anatomical structures between the two modalities during the synthesis. A cross-modal attention scheme is proposed to leverage non-local spatial correlations. The feasibility of the approach to produce realistic looking MR images is demonstrated quantitatively and with a qualitative evaluation compared to real fetal MR images. |
| spellingShingle | Jiao, J Namburete, AIL Papageorghiou, AT Noble, JA Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title | Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title_full | Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title_fullStr | Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title_full_unstemmed | Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title_short | Anatomy-aware self-supervised fetal MRI synthesis from unpaired ultrasound images |
| title_sort | anatomy aware self supervised fetal mri synthesis from unpaired ultrasound images |
| work_keys_str_mv | AT jiaoj anatomyawareselfsupervisedfetalmrisynthesisfromunpairedultrasoundimages AT nambureteail anatomyawareselfsupervisedfetalmrisynthesisfromunpairedultrasoundimages AT papageorghiouat anatomyawareselfsupervisedfetalmrisynthesisfromunpairedultrasoundimages AT nobleja anatomyawareselfsupervisedfetalmrisynthesisfromunpairedultrasoundimages |