Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation
<p><strong>Purpose:</strong> We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not...
| Main Authors: | , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Society of Photo-optical Instrumentation Engineers
2020
|
| _version_ | 1797056731841298432 |
|---|---|
| author | Xu, Y Lee, LH Drukker, L Yaqub, M Papageorghiou, AT Noble, AJ |
| author_facet | Xu, Y Lee, LH Drukker, L Yaqub, M Papageorghiou, AT Noble, AJ |
| author_sort | Xu, Y |
| collection | OXFORD |
| description | <p><strong>Purpose:</strong> We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not available. We also report original insights on the general appearance differences between second- and third-trimester fetal head transventricular (TV) plane images.</p>
<p><strong>Approach:</strong> We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second- and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deep-learning-based image recognition algorithm (ScanNav®) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently.</p>
<p><strong>Results:</strong> Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second- and real third-trimester scans was computed by ScanNav® and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method.</p>
<p><strong>Conclusions:</strong> The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav®) for large-scale quantitative image analysis evaluation is also a first to our knowledge.</p> |
| first_indexed | 2024-03-06T19:26:45Z |
| format | Journal article |
| id | oxford-uuid:1bff0bbb-599d-4960-8e8a-51d70ce3ec0e |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T19:26:45Z |
| publishDate | 2020 |
| publisher | Society of Photo-optical Instrumentation Engineers |
| record_format | dspace |
| spelling | oxford-uuid:1bff0bbb-599d-4960-8e8a-51d70ce3ec0e2022-03-26T11:03:33ZSimulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluationJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:1bff0bbb-599d-4960-8e8a-51d70ce3ec0eEnglishSymplectic ElementsSociety of Photo-optical Instrumentation Engineers2020Xu, YLee, LHDrukker, LYaqub, MPapageorghiou, ATNoble, AJ<p><strong>Purpose:</strong> We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not available. We also report original insights on the general appearance differences between second- and third-trimester fetal head transventricular (TV) plane images.</p> <p><strong>Approach:</strong> We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second- and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deep-learning-based image recognition algorithm (ScanNav®) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently.</p> <p><strong>Results:</strong> Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second- and real third-trimester scans was computed by ScanNav® and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method.</p> <p><strong>Conclusions:</strong> The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav®) for large-scale quantitative image analysis evaluation is also a first to our knowledge.</p> |
| spellingShingle | Xu, Y Lee, LH Drukker, L Yaqub, M Papageorghiou, AT Noble, AJ Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title | Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title_full | Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title_fullStr | Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title_full_unstemmed | Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title_short | Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation |
| title_sort | simulating realistic fetal neurosonography images with appearance and growth change using cycle consistent adversarial networks and an evaluation |
| work_keys_str_mv | AT xuy simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation AT leelh simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation AT drukkerl simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation AT yaqubm simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation AT papageorghiouat simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation AT nobleaj simulatingrealisticfetalneurosonographyimageswithappearanceandgrowthchangeusingcycleconsistentadversarialnetworksandanevaluation |