Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training
Background: Surgical education lacks a standardized, proficiency-based approach to evaluation and feedback. Objective: To assess the implementation and reception (ie, feasibility) of an automated, standardized, longitudinal surgical skill assessment and feedback system, and identify baseline trainee...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-04-01
|
| Series: | European Urology Open Science |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666168324002714 |
| _version_ | 1797235441310629888 |
|---|---|
| author | Rand N. Wilcox Vanden Berg Emily A. Vertosick Daniel D. Sjoberg Eugene K. Cha Jonathan A. Coleman Timothy F. Donahue James A. Eastham Behfar Ehdaie Vincent P. Laudone Eugene J. Pietzak Robert C. Smith Alvin C. Goh |
| author_facet | Rand N. Wilcox Vanden Berg Emily A. Vertosick Daniel D. Sjoberg Eugene K. Cha Jonathan A. Coleman Timothy F. Donahue James A. Eastham Behfar Ehdaie Vincent P. Laudone Eugene J. Pietzak Robert C. Smith Alvin C. Goh |
| author_sort | Rand N. Wilcox Vanden Berg |
| collection | DOAJ |
| description | Background: Surgical education lacks a standardized, proficiency-based approach to evaluation and feedback. Objective: To assess the implementation and reception (ie, feasibility) of an automated, standardized, longitudinal surgical skill assessment and feedback system, and identify baseline trainee (resident and fellow) characteristics associated with achieving proficiency in robotic surgery while learning robotic-assisted laparoscopic prostatectomy. Design, setting, and participants: A quality improvement study assessing a pilot of a surgical experience tracking program was conducted over 1 yr. Participants were six fellows, eight residents, and nine attending surgeons at a tertiary cancer center. Intervention: Trainees underwent baseline self-assessment. After each surgery, an evaluation was completed independently by the trainee and attending surgeons. Performance was rated on a five-point anchored Likert scale (trainees were considered “proficient” when attending surgeons’ rating was ≥4). Technical skills were assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) and Prostatectomy Assessment and Competency Evaluation (PACE). Outcome measurements and statistical analysis: Program success and utility were assessed by evaluating completion rates, evaluation completion times, and concordance rates between attending and trainee surgeons, and exit surveys. Baseline characteristics were assessed to determine associations with achieving proficiency. Results and limitations: Completion rates for trainees and attending surgeons were 72% and 77%, respectively. Fellows performed more steps/cases than residents (median [interquartile range]: 5 [3–7] and 3 [2–4], respectively; p < 0.01). Prior completion of robotics or laparoscopic skill courses and surgical experience measures were associated with achieving proficiency in multiple surgical steps and GEARS domains. Interclass correlation coefficients on individual components were 0.27–0.47 on GEARS domains. Conclusions: An automated surgical experience tracker with structured, longitudinal evaluation and feedback can be implemented with good participation and minimal participant time commitment, and can guide curricular development in a proficiency-based education program by identifying modifiable factors associated with proficiency, individualizing education, and identifying improvement areas within the education program. Patient summary: An automated, standardized, longitudinal surgical skill assessment and feedback system can be implemented successfully in surgical education settings and used to inform education plans and predict trainee proficiency. |
| first_indexed | 2024-03-07T13:58:45Z |
| format | Article |
| id | doaj.art-0818a6fe6d524cd99f3ec36c5ed129cd |
| institution | Directory Open Access Journal |
| issn | 2666-1683 |
| language | English |
| last_indexed | 2024-04-24T16:48:00Z |
| publishDate | 2024-04-01 |
| publisher | Elsevier |
| record_format | Article |
| series | European Urology Open Science |
| spelling | doaj.art-0818a6fe6d524cd99f3ec36c5ed129cd2024-03-29T05:51:13ZengElsevierEuropean Urology Open Science2666-16832024-04-01628190Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on TrainingRand N. Wilcox Vanden Berg0Emily A. Vertosick1Daniel D. Sjoberg2Eugene K. Cha3Jonathan A. Coleman4Timothy F. Donahue5James A. Eastham6Behfar Ehdaie7Vincent P. Laudone8Eugene J. Pietzak9Robert C. Smith10Alvin C. Goh11Urology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USADepartment of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USADepartment of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USAUrology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Corresponding author. Memorial Sloan Kettering Cancer Center, 353 E. 68th St, K512, New York, NY 10065, USA. Tel. +1 (646) 422-4667; Fax: +1 (212) 988-0760.Background: Surgical education lacks a standardized, proficiency-based approach to evaluation and feedback. Objective: To assess the implementation and reception (ie, feasibility) of an automated, standardized, longitudinal surgical skill assessment and feedback system, and identify baseline trainee (resident and fellow) characteristics associated with achieving proficiency in robotic surgery while learning robotic-assisted laparoscopic prostatectomy. Design, setting, and participants: A quality improvement study assessing a pilot of a surgical experience tracking program was conducted over 1 yr. Participants were six fellows, eight residents, and nine attending surgeons at a tertiary cancer center. Intervention: Trainees underwent baseline self-assessment. After each surgery, an evaluation was completed independently by the trainee and attending surgeons. Performance was rated on a five-point anchored Likert scale (trainees were considered “proficient” when attending surgeons’ rating was ≥4). Technical skills were assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) and Prostatectomy Assessment and Competency Evaluation (PACE). Outcome measurements and statistical analysis: Program success and utility were assessed by evaluating completion rates, evaluation completion times, and concordance rates between attending and trainee surgeons, and exit surveys. Baseline characteristics were assessed to determine associations with achieving proficiency. Results and limitations: Completion rates for trainees and attending surgeons were 72% and 77%, respectively. Fellows performed more steps/cases than residents (median [interquartile range]: 5 [3–7] and 3 [2–4], respectively; p < 0.01). Prior completion of robotics or laparoscopic skill courses and surgical experience measures were associated with achieving proficiency in multiple surgical steps and GEARS domains. Interclass correlation coefficients on individual components were 0.27–0.47 on GEARS domains. Conclusions: An automated surgical experience tracker with structured, longitudinal evaluation and feedback can be implemented with good participation and minimal participant time commitment, and can guide curricular development in a proficiency-based education program by identifying modifiable factors associated with proficiency, individualizing education, and identifying improvement areas within the education program. Patient summary: An automated, standardized, longitudinal surgical skill assessment and feedback system can be implemented successfully in surgical education settings and used to inform education plans and predict trainee proficiency.http://www.sciencedirect.com/science/article/pii/S2666168324002714EducationLearning curveRobotics |
| spellingShingle | Rand N. Wilcox Vanden Berg Emily A. Vertosick Daniel D. Sjoberg Eugene K. Cha Jonathan A. Coleman Timothy F. Donahue James A. Eastham Behfar Ehdaie Vincent P. Laudone Eugene J. Pietzak Robert C. Smith Alvin C. Goh Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training European Urology Open Science Education Learning curve Robotics |
| title | Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training |
| title_full | Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training |
| title_fullStr | Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training |
| title_full_unstemmed | Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training |
| title_short | Implementation and Validation of an Automated, Longitudinal Robotic Surgical Evaluation and Feedback Program at a High-volume Center and Impact on Training |
| title_sort | implementation and validation of an automated longitudinal robotic surgical evaluation and feedback program at a high volume center and impact on training |
| topic | Education Learning curve Robotics |
| url | http://www.sciencedirect.com/science/article/pii/S2666168324002714 |
| work_keys_str_mv | AT randnwilcoxvandenberg implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT emilyavertosick implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT danieldsjoberg implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT eugenekcha implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT jonathanacoleman implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT timothyfdonahue implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT jamesaeastham implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT behfarehdaie implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT vincentplaudone implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT eugenejpietzak implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT robertcsmith implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining AT alvincgoh implementationandvalidationofanautomatedlongitudinalroboticsurgicalevaluationandfeedbackprogramatahighvolumecenterandimpactontraining |