Advancing ultrasound image analysis by capturing operator gaze patterns
<p>Obstetric ultrasound scanning is a safe and effective tool for the early detection of fetal abnormalities and therefore crucial for determining the necessity of clinical intervention. However, ultrasound relies on operator expertise, which is a scarce resource globally. Moreover, there are...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2021
|
| Subjects: |
| _version_ | 1826280758558326784 |
|---|---|
| author | Droste, R |
| author2 | Noble, A |
| author_facet | Noble, A Droste, R |
| author_sort | Droste, R |
| collection | OXFORD |
| description | <p>Obstetric ultrasound scanning is a safe and effective tool for the early detection of fetal abnormalities and therefore crucial for determining the necessity of clinical intervention. However, ultrasound relies on operator expertise, which is a scarce resource globally. Moreover, there are large geographical and inter-observer variations of clinical outcomes. To address this, the PULSE project aims to develop a new generation of ultrasound scanning capabilities based on big data and machine learning models which capture the knowledge of experienced sonographers. To this end, the project team acquired a first-of-its-kind large-scale dataset of routine clinical ultrasound scanning with gaze-tracking data.</p>
<p>In this thesis, we first examine shortcomings of the operator-machine interaction. We find that sonographers adjust the biometric measurements of fetuses with potential growth abnormalities towards the healthy expected value, providing a possible explanation for the known deficiencies of these measurements. Moreover, we study the adherence to safety recommendations regarding thermal energy emission and find that, while sonographers keep within the appropriate limits, they rarely check the safety indices. We provide suggestions for the modification of the ultrasound machine interface to address these two issues.</p>
<p>Second, we develop the first model that predicts sonographer gaze-tracking data on ultrasound video through the method of visual saliency prediction. In addition, we propose the first unified visual saliency model for the prediction of gaze on both images and videos. Besides unifying the two modalities, the model obtains state-of-the-art performance on both tasks for all relevant computer vision benchmarks.</p>
<p>Third, we show that sonographer gaze-tracking data is a powerful supervision signal for ultrasound image feature representation learning. We develop a general framework for representation learning and transfer of the trained neural network to the downstream tasks of standard plane detection and automatic biometry plane annotation. We also show that the learned representations, in combination with the sonographer gaze prediction, can be used to discover and localize visually salient anatomical landmarks, i.e., landmarks that sonographers use for visual navigation.</p>
<p>Finally, we provide an overarching discussion and an extended outlook chapter which describes a system for guiding sonographers during standard plane acquisition.</p> |
| first_indexed | 2024-03-07T00:18:31Z |
| format | Thesis |
| id | oxford-uuid:7bb0d0fc-8a52-45a2-8fff-f7aff014b035 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T00:18:31Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | oxford-uuid:7bb0d0fc-8a52-45a2-8fff-f7aff014b0352022-03-26T20:52:17ZAdvancing ultrasound image analysis by capturing operator gaze patternsThesishttp://purl.org/coar/resource_type/c_db06uuid:7bb0d0fc-8a52-45a2-8fff-f7aff014b035Fetus--Ultrasonic imagingMachine learningUltrasonics in obstetricsNeural networks (Computer science)Computer vision in medicineArtificial intelligenceEye trackingEnglishHyrax Deposit2021Droste, RNoble, AGrau, VRückert, D<p>Obstetric ultrasound scanning is a safe and effective tool for the early detection of fetal abnormalities and therefore crucial for determining the necessity of clinical intervention. However, ultrasound relies on operator expertise, which is a scarce resource globally. Moreover, there are large geographical and inter-observer variations of clinical outcomes. To address this, the PULSE project aims to develop a new generation of ultrasound scanning capabilities based on big data and machine learning models which capture the knowledge of experienced sonographers. To this end, the project team acquired a first-of-its-kind large-scale dataset of routine clinical ultrasound scanning with gaze-tracking data.</p> <p>In this thesis, we first examine shortcomings of the operator-machine interaction. We find that sonographers adjust the biometric measurements of fetuses with potential growth abnormalities towards the healthy expected value, providing a possible explanation for the known deficiencies of these measurements. Moreover, we study the adherence to safety recommendations regarding thermal energy emission and find that, while sonographers keep within the appropriate limits, they rarely check the safety indices. We provide suggestions for the modification of the ultrasound machine interface to address these two issues.</p> <p>Second, we develop the first model that predicts sonographer gaze-tracking data on ultrasound video through the method of visual saliency prediction. In addition, we propose the first unified visual saliency model for the prediction of gaze on both images and videos. Besides unifying the two modalities, the model obtains state-of-the-art performance on both tasks for all relevant computer vision benchmarks.</p> <p>Third, we show that sonographer gaze-tracking data is a powerful supervision signal for ultrasound image feature representation learning. We develop a general framework for representation learning and transfer of the trained neural network to the downstream tasks of standard plane detection and automatic biometry plane annotation. We also show that the learned representations, in combination with the sonographer gaze prediction, can be used to discover and localize visually salient anatomical landmarks, i.e., landmarks that sonographers use for visual navigation.</p> <p>Finally, we provide an overarching discussion and an extended outlook chapter which describes a system for guiding sonographers during standard plane acquisition.</p> |
| spellingShingle | Fetus--Ultrasonic imaging Machine learning Ultrasonics in obstetrics Neural networks (Computer science) Computer vision in medicine Artificial intelligence Eye tracking Droste, R Advancing ultrasound image analysis by capturing operator gaze patterns |
| title | Advancing ultrasound image analysis by capturing operator gaze patterns |
| title_full | Advancing ultrasound image analysis by capturing operator gaze patterns |
| title_fullStr | Advancing ultrasound image analysis by capturing operator gaze patterns |
| title_full_unstemmed | Advancing ultrasound image analysis by capturing operator gaze patterns |
| title_short | Advancing ultrasound image analysis by capturing operator gaze patterns |
| title_sort | advancing ultrasound image analysis by capturing operator gaze patterns |
| topic | Fetus--Ultrasonic imaging Machine learning Ultrasonics in obstetrics Neural networks (Computer science) Computer vision in medicine Artificial intelligence Eye tracking |
| work_keys_str_mv | AT droster advancingultrasoundimageanalysisbycapturingoperatorgazepatterns |