Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations
As the number of robotic surgery procedures has increased, so has the importance of evaluating surgical skills in these techniques. It is difficult, however, to automatically and quantitatively evaluate surgical skills during robotic surgery, as these skills are primarily associated with the movemen...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-06-01
|
| Series: | Journal of Clinical Medicine |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0383/9/6/1964 |
| _version_ | 1797564361575759872 |
|---|---|
| author | Dongheon Lee Hyeong Won Yu Hyungju Kwon Hyoun-Joong Kong Kyu Eun Lee Hee Chan Kim |
| author_facet | Dongheon Lee Hyeong Won Yu Hyungju Kwon Hyoun-Joong Kong Kyu Eun Lee Hee Chan Kim |
| author_sort | Dongheon Lee |
| collection | DOAJ |
| description | As the number of robotic surgery procedures has increased, so has the importance of evaluating surgical skills in these techniques. It is difficult, however, to automatically and quantitatively evaluate surgical skills during robotic surgery, as these skills are primarily associated with the movement of surgical instruments. This study proposes a deep learning-based surgical instrument tracking algorithm to evaluate surgeons’ skills in performing procedures by robotic surgery. This method overcame two main drawbacks: occlusion and maintenance of the identity of the surgical instruments. In addition, surgical skill prediction models were developed using motion metrics calculated from the motion of the instruments. The tracking method was applied to 54 video segments and evaluated by root mean squared error (RMSE), area under the curve (AUC), and Pearson correlation analysis. The RMSE was 3.52 mm, the AUC of 1 mm, 2 mm, and 5 mm were 0.7, 0.78, and 0.86, respectively, and Pearson’s correlation coefficients were 0.9 on the <i>x</i>-axis and 0.87 on the <i>y</i>-axis. The surgical skill prediction models showed an accuracy of 83% with Objective Structured Assessment of Technical Skill (OSATS) and Global Evaluative Assessment of Robotic Surgery (GEARS). The proposed method was able to track instruments during robotic surgery, suggesting that the current method of surgical skill assessment by surgeons can be replaced by the proposed automatic and quantitative evaluation method. |
| first_indexed | 2024-03-10T18:56:16Z |
| format | Article |
| id | doaj.art-5beb354ec1184b0897e30bb2adb7595d |
| institution | Directory Open Access Journal |
| issn | 2077-0383 |
| language | English |
| last_indexed | 2024-03-10T18:56:16Z |
| publishDate | 2020-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Clinical Medicine |
| spelling | doaj.art-5beb354ec1184b0897e30bb2adb7595d2023-11-20T04:43:25ZengMDPI AGJournal of Clinical Medicine2077-03832020-06-0196196410.3390/jcm9061964Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual OperationsDongheon Lee0Hyeong Won Yu1Hyungju Kwon2Hyoun-Joong Kong3Kyu Eun Lee4Hee Chan Kim5Interdisciplinary Program, Bioengineering Major, Graduate School, Seoul National University, 101 Daehak-ro, Jongno-gu, Seoul 03080, KoreaDepartment of Surgery, Seoul National University Bundang Hospital, 82, Gumi-ro 173 Beon-gil, Bundang-gu, Seongnam-si, Gyeonggi-do 13620, KoreaDepartment of Surgery, Ewha Womans University Medical Center, 1071 Anyangcheon-ro, Yangcheon-Gu, Seoul 07985, KoreaDepartment of Biomedical Engineering, Chungnam National University Hospital & College of Medicine, 282 Munhwa-ro, Jung-gu, Daejeon 301-721, KoreaDepartment of Surgery, Seoul National University Hospital and College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, KoreaInstitute of Medical & Biological Engineering, Medical Research Center, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, KoreaAs the number of robotic surgery procedures has increased, so has the importance of evaluating surgical skills in these techniques. It is difficult, however, to automatically and quantitatively evaluate surgical skills during robotic surgery, as these skills are primarily associated with the movement of surgical instruments. This study proposes a deep learning-based surgical instrument tracking algorithm to evaluate surgeons’ skills in performing procedures by robotic surgery. This method overcame two main drawbacks: occlusion and maintenance of the identity of the surgical instruments. In addition, surgical skill prediction models were developed using motion metrics calculated from the motion of the instruments. The tracking method was applied to 54 video segments and evaluated by root mean squared error (RMSE), area under the curve (AUC), and Pearson correlation analysis. The RMSE was 3.52 mm, the AUC of 1 mm, 2 mm, and 5 mm were 0.7, 0.78, and 0.86, respectively, and Pearson’s correlation coefficients were 0.9 on the <i>x</i>-axis and 0.87 on the <i>y</i>-axis. The surgical skill prediction models showed an accuracy of 83% with Objective Structured Assessment of Technical Skill (OSATS) and Global Evaluative Assessment of Robotic Surgery (GEARS). The proposed method was able to track instruments during robotic surgery, suggesting that the current method of surgical skill assessment by surgeons can be replaced by the proposed automatic and quantitative evaluation method.https://www.mdpi.com/2077-0383/9/6/1964surgical skillsrobotic surgerydeep learningsurgical instrument trackingquantitative evaluation |
| spellingShingle | Dongheon Lee Hyeong Won Yu Hyungju Kwon Hyoun-Joong Kong Kyu Eun Lee Hee Chan Kim Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations Journal of Clinical Medicine surgical skills robotic surgery deep learning surgical instrument tracking quantitative evaluation |
| title | Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations |
| title_full | Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations |
| title_fullStr | Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations |
| title_full_unstemmed | Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations |
| title_short | Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations |
| title_sort | evaluation of surgical skills during robotic surgery by deep learning based multiple surgical instrument tracking in training and actual operations |
| topic | surgical skills robotic surgery deep learning surgical instrument tracking quantitative evaluation |
| url | https://www.mdpi.com/2077-0383/9/6/1964 |
| work_keys_str_mv | AT dongheonlee evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations AT hyeongwonyu evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations AT hyungjukwon evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations AT hyounjoongkong evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations AT kyueunlee evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations AT heechankim evaluationofsurgicalskillsduringroboticsurgerybydeeplearningbasedmultiplesurgicalinstrumenttrackingintrainingandactualoperations |