Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm
Objective:. Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient-boosting classification model to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer Health
2023-06-01
|
| Series: | Annals of Surgery Open |
| Online Access: | http://journals.lww.com/10.1097/AS9.0000000000000292 |
| _version_ | 1797732787879411712 |
|---|---|
| author | Somayeh B. Shafiei Saeed Shadpour James L. Mohler Kristopher Attwood Qian Liu Camille Gutierrez Mehdi Seilanian Toussi |
| author_facet | Somayeh B. Shafiei Saeed Shadpour James L. Mohler Kristopher Attwood Qian Liu Camille Gutierrez Mehdi Seilanian Toussi |
| author_sort | Somayeh B. Shafiei |
| collection | DOAJ |
| description | Objective:. Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient-boosting classification model to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted surgery (RAS) using visual metrics.
Methods:. Eye gaze data were recorded from 11 participants performing 4 subtasks; blunt dissection, retraction, cold dissection, and hot dissection using live pigs and the da Vinci robot. Eye gaze data were used to extract the visual metrics. One expert RAS surgeon evaluated each participant’s performance and expertise level using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The extracted visual metrics were used to classify surgical skill levels and to evaluate individual GEARS metrics. Analysis of Variance (ANOVA) was used to test the differences for each feature across skill levels.
Results:. Classification accuracies for blunt dissection, retraction, cold dissection, and burn dissection were 95%, 96%, 96%, and 96%, respectively. The time to complete only the retraction was significantly different among the 3 skill levels (P value = 0.04). Performance was significantly different for 3 categories of surgical skill level for all subtasks (P values < 0.01). The extracted visual metrics were strongly associated with GEARS metrics (R2 > 0.7 for GEARS metrics evaluation models).
Conclusions:. Machine learning algorithms trained by visual metrics of RAS surgeons can classify surgical skill levels and evaluate GEARS measures. The time to complete a surgical subtask may not be considered a stand-alone factor for skill level assessment. |
| first_indexed | 2024-03-12T12:18:38Z |
| format | Article |
| id | doaj.art-562181044b6b4a199ac8473d01ace7aa |
| institution | Directory Open Access Journal |
| issn | 2691-3593 |
| language | English |
| last_indexed | 2024-03-12T12:18:38Z |
| publishDate | 2023-06-01 |
| publisher | Wolters Kluwer Health |
| record_format | Article |
| series | Annals of Surgery Open |
| spelling | doaj.art-562181044b6b4a199ac8473d01ace7aa2023-08-30T06:11:03ZengWolters Kluwer HealthAnnals of Surgery Open2691-35932023-06-0142e29210.1097/AS9.0000000000000292202306000-00016Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting AlgorithmSomayeh B. Shafiei0Saeed Shadpour1James L. Mohler2Kristopher Attwood3Qian Liu4Camille Gutierrez5Mehdi Seilanian Toussi6From the * Department of Urology, Roswell Park Comprehensive Cancer Center in Buffalo, NY† Department of Animal Biosciences, University of Guelph, Guelph, Ontario, CanadaFrom the * Department of Urology, Roswell Park Comprehensive Cancer Center in Buffalo, NY‡ Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY‡ Department of Biostatistics and Bioinformatics, Roswell Park Comprehensive Cancer Center, Buffalo, NY§ Obstetrics and Gynecology Residency Program, Sisters of Charity Health System, Buffalo, NY.From the * Department of Urology, Roswell Park Comprehensive Cancer Center in Buffalo, NYObjective:. Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient-boosting classification model to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted surgery (RAS) using visual metrics. Methods:. Eye gaze data were recorded from 11 participants performing 4 subtasks; blunt dissection, retraction, cold dissection, and hot dissection using live pigs and the da Vinci robot. Eye gaze data were used to extract the visual metrics. One expert RAS surgeon evaluated each participant’s performance and expertise level using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The extracted visual metrics were used to classify surgical skill levels and to evaluate individual GEARS metrics. Analysis of Variance (ANOVA) was used to test the differences for each feature across skill levels. Results:. Classification accuracies for blunt dissection, retraction, cold dissection, and burn dissection were 95%, 96%, 96%, and 96%, respectively. The time to complete only the retraction was significantly different among the 3 skill levels (P value = 0.04). Performance was significantly different for 3 categories of surgical skill level for all subtasks (P values < 0.01). The extracted visual metrics were strongly associated with GEARS metrics (R2 > 0.7 for GEARS metrics evaluation models). Conclusions:. Machine learning algorithms trained by visual metrics of RAS surgeons can classify surgical skill levels and evaluate GEARS measures. The time to complete a surgical subtask may not be considered a stand-alone factor for skill level assessment.http://journals.lww.com/10.1097/AS9.0000000000000292 |
| spellingShingle | Somayeh B. Shafiei Saeed Shadpour James L. Mohler Kristopher Attwood Qian Liu Camille Gutierrez Mehdi Seilanian Toussi Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm Annals of Surgery Open |
| title | Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm |
| title_full | Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm |
| title_fullStr | Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm |
| title_full_unstemmed | Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm |
| title_short | Developing Surgical Skill Level Classification Model Using Visual Metrics and a Gradient Boosting Algorithm |
| title_sort | developing surgical skill level classification model using visual metrics and a gradient boosting algorithm |
| url | http://journals.lww.com/10.1097/AS9.0000000000000292 |
| work_keys_str_mv | AT somayehbshafiei developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT saeedshadpour developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT jameslmohler developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT kristopherattwood developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT qianliu developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT camillegutierrez developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm AT mehdiseilaniantoussi developingsurgicalskilllevelclassificationmodelusingvisualmetricsandagradientboostingalgorithm |