Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography
There have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, l...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-01-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2021.735177/full |
| _version_ | 1819364797968285696 |
|---|---|
| author | Min-seok Kim Joon Hyuk Cha Seonhwa Lee Lihong Han Lihong Han Wonhyoung Park Jae Sung Ahn Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park |
| author_facet | Min-seok Kim Joon Hyuk Cha Seonhwa Lee Lihong Han Lihong Han Wonhyoung Park Jae Sung Ahn Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park |
| author_sort | Min-seok Kim |
| collection | DOAJ |
| description | There have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, large deformations, occlusions, low illumination, and inadequate focus occur. In addition, it is difficult and costly to obtain a large and accurate dataset on operational video anatomical structures, including arteries. In this study, we investigated cerebral artery segmentation using an automatic ground-truth generation method. Indocyanine green (ICG) fluorescence intraoperative cerebral videoangiography was used to create a ground-truth dataset mainly for cerebral arteries and partly for cerebral blood vessels, including veins. Four different neural network models were trained using the dataset and compared. Before augmentation, 35,975 training images and 11,266 validation images were used. After augmentation, 260,499 training and 90,129 validation images were used. A Dice score of 79% for cerebral artery segmentation was achieved using the DeepLabv3+ model trained using an automatically generated dataset. Strict validation in different patient groups was conducted. Arteries were also discerned from the veins using the ICG videoangiography phase. We achieved fair accuracy, which demonstrated the appropriateness of the methodology. This study proved the feasibility of operating field view of the cerebral artery segmentation using deep learning, and the effectiveness of the automatic blood vessel ground truth generation method using ICG fluorescence videoangiography. Using this method, computer vision can discern blood vessels and arteries from veins in a neurosurgical microscope field of view. Thus, this technique is essential for neurosurgical field vessel anatomy-based navigation. In addition, surgical assistance, safety, and autonomous surgery neurorobotics that can detect or manipulate cerebral vessels would require computer vision to identify blood vessels and arteries. |
| first_indexed | 2024-12-24T23:04:39Z |
| format | Article |
| id | doaj.art-178212b3545e46d1a004b9c8df94736c |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-12-24T23:04:39Z |
| publishDate | 2022-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-178212b3545e46d1a004b9c8df94736c2022-12-21T16:35:03ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182022-01-011510.3389/fnbot.2021.735177735177Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence VideoangiographyMin-seok Kim0Joon Hyuk Cha1Seonhwa Lee2Lihong Han3Lihong Han4Wonhyoung Park5Jae Sung Ahn6Seong-Cheol Park7Seong-Cheol Park8Seong-Cheol Park9Seong-Cheol Park10Clinical Research Team, Deepnoid, Seoul, South KoreaDepartment of Internal Medicine, Inha University Hospital, Incheon, South KoreaDepartment of Bio-convergence Engineering, Korea University, Seoul, South KoreaClinical Research Team, Deepnoid, Seoul, South KoreaDepartment of Computer Science and Engineering, Soongsil University, Seoul, South KoreaDepartment of Neurosurgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South KoreaDepartment of Neurosurgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South KoreaClinical Research Team, Deepnoid, Seoul, South KoreaDepartment of Neurosurgery, Gangneung Asan Hospital, University of Ulsan College of Medicine, Gangneung, South KoreaDepartment of Neurosurgery, Seoul Metropolitan Government—Seoul National University Boramae Medical Center, Seoul, South KoreaDepartment of Neurosurgery, Hallym Hospital, Incheon, South KoreaThere have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, large deformations, occlusions, low illumination, and inadequate focus occur. In addition, it is difficult and costly to obtain a large and accurate dataset on operational video anatomical structures, including arteries. In this study, we investigated cerebral artery segmentation using an automatic ground-truth generation method. Indocyanine green (ICG) fluorescence intraoperative cerebral videoangiography was used to create a ground-truth dataset mainly for cerebral arteries and partly for cerebral blood vessels, including veins. Four different neural network models were trained using the dataset and compared. Before augmentation, 35,975 training images and 11,266 validation images were used. After augmentation, 260,499 training and 90,129 validation images were used. A Dice score of 79% for cerebral artery segmentation was achieved using the DeepLabv3+ model trained using an automatically generated dataset. Strict validation in different patient groups was conducted. Arteries were also discerned from the veins using the ICG videoangiography phase. We achieved fair accuracy, which demonstrated the appropriateness of the methodology. This study proved the feasibility of operating field view of the cerebral artery segmentation using deep learning, and the effectiveness of the automatic blood vessel ground truth generation method using ICG fluorescence videoangiography. Using this method, computer vision can discern blood vessels and arteries from veins in a neurosurgical microscope field of view. Thus, this technique is essential for neurosurgical field vessel anatomy-based navigation. In addition, surgical assistance, safety, and autonomous surgery neurorobotics that can detect or manipulate cerebral vessels would require computer vision to identify blood vessels and arteries.https://www.frontiersin.org/articles/10.3389/fnbot.2021.735177/fullsemantic segmentationneural networkblood vesselindocyanine greenneurosurgical fieldcomputer vision |
| spellingShingle | Min-seok Kim Joon Hyuk Cha Seonhwa Lee Lihong Han Lihong Han Wonhyoung Park Jae Sung Ahn Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park Seong-Cheol Park Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography Frontiers in Neurorobotics semantic segmentation neural network blood vessel indocyanine green neurosurgical field computer vision |
| title | Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography |
| title_full | Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography |
| title_fullStr | Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography |
| title_full_unstemmed | Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography |
| title_short | Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography |
| title_sort | deep learning based cerebral artery semantic segmentation in neurosurgical operating microscope vision using indocyanine green fluorescence videoangiography |
| topic | semantic segmentation neural network blood vessel indocyanine green neurosurgical field computer vision |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2021.735177/full |
| work_keys_str_mv | AT minseokkim deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT joonhyukcha deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT seonhwalee deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT lihonghan deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT lihonghan deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT wonhyoungpark deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT jaesungahn deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT seongcheolpark deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT seongcheolpark deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT seongcheolpark deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography AT seongcheolpark deeplearningbasedcerebralarterysemanticsegmentationinneurosurgicaloperatingmicroscopevisionusingindocyaninegreenfluorescencevideoangiography |