Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework
Abstract Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomica...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-11-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-45466-9 |
| _version_ | 1827770613278703616 |
|---|---|
| author | Jannik Stebani Martin Blaimer Simon Zabler Tilmann Neun Daniël M. Pelt Kristen Rak |
| author_facet | Jannik Stebani Martin Blaimer Simon Zabler Tilmann Neun Daniël M. Pelt Kristen Rak |
| author_sort | Jannik Stebani |
| collection | DOAJ |
| description | Abstract Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ( $$N=43$$ N = 43 ) and clinical practice ( $$N=9$$ N = 9 ). The model robustness was further evaluated on three independent open-source datasets ( $$N = 23{} + 7{} + 17$$ N = 23 + 7 + 17 scans) consisting of cadaveric specimen scans. For the in-house datasets, Dice scores of $$\text{0.97 and 0.94}$$ 0.97 and 0.94 , intersection-over-union scores of $$\text{0.94 and 0.89}$$ 0.94 and 0.89 and average Hausdorff distances of $$0.065{}$$ 0.065 and $$0.14{}$$ 0.14 voxel units were achieved. The landmark localization task was performed automatically with an average localization error of $$\text{3.3 and 5.2}$$ 3.3 and 5.2 voxel units. A robust, albeit reduced performance could be attained for the catalogue of three open-source datasets. Results of the ablation studies with 43 mono-parametric variations of the basal architecture and training protocol provided task-optimal parameters for both categories. Ablation studies against single-task variants of the basal architecture showed a clear performance benefit of coupling landmark localization with segmentation and a dataset-dependent performance impact on segmentation ability. |
| first_indexed | 2024-03-11T12:42:03Z |
| format | Article |
| id | doaj.art-c1cb53b813d64df08b5bc216ab140e33 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-11T12:42:03Z |
| publishDate | 2023-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-c1cb53b813d64df08b5bc216ab140e332023-11-05T12:15:32ZengNature PortfolioScientific Reports2045-23222023-11-0113111610.1038/s41598-023-45466-9Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection frameworkJannik Stebani0Martin Blaimer1Simon Zabler2Tilmann Neun3Daniël M. Pelt4Kristen Rak5Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IISMagnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IISMagnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IISInstitute for Diagnostic and Interventional Neuroradiology, Universitätsklinikum WürzburgLeiden Institute of Advanced Computer Science (LIACS), Universiteit LeidenDepartment of Oto-Rhino-Laryngology, Plastic, Aesthetic and Reconstructive Head and Neck Surgery and the Comprehensive Hearing Center, Universitätsklinikum WürzburgAbstract Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ( $$N=43$$ N = 43 ) and clinical practice ( $$N=9$$ N = 9 ). The model robustness was further evaluated on three independent open-source datasets ( $$N = 23{} + 7{} + 17$$ N = 23 + 7 + 17 scans) consisting of cadaveric specimen scans. For the in-house datasets, Dice scores of $$\text{0.97 and 0.94}$$ 0.97 and 0.94 , intersection-over-union scores of $$\text{0.94 and 0.89}$$ 0.94 and 0.89 and average Hausdorff distances of $$0.065{}$$ 0.065 and $$0.14{}$$ 0.14 voxel units were achieved. The landmark localization task was performed automatically with an average localization error of $$\text{3.3 and 5.2}$$ 3.3 and 5.2 voxel units. A robust, albeit reduced performance could be attained for the catalogue of three open-source datasets. Results of the ablation studies with 43 mono-parametric variations of the basal architecture and training protocol provided task-optimal parameters for both categories. Ablation studies against single-task variants of the basal architecture showed a clear performance benefit of coupling landmark localization with segmentation and a dataset-dependent performance impact on segmentation ability.https://doi.org/10.1038/s41598-023-45466-9 |
| spellingShingle | Jannik Stebani Martin Blaimer Simon Zabler Tilmann Neun Daniël M. Pelt Kristen Rak Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework Scientific Reports |
| title | Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework |
| title_full | Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework |
| title_fullStr | Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework |
| title_full_unstemmed | Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework |
| title_short | Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework |
| title_sort | towards fully automated inner ear analysis with deep learning based joint segmentation and landmark detection framework |
| url | https://doi.org/10.1038/s41598-023-45466-9 |
| work_keys_str_mv | AT jannikstebani towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework AT martinblaimer towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework AT simonzabler towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework AT tilmannneun towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework AT danielmpelt towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework AT kristenrak towardsfullyautomatedinnerearanalysiswithdeeplearningbasedjointsegmentationandlandmarkdetectionframework |