A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System
This paper describes a recording and playback system developed using a da Vinci Standard Surgical System and research kit. The system records stereo laparoscopic videos, robot arm joint angles, and surgeon⁻console interactions in a synchronized manner. A user can then, on-demand and at adj...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-02-01
|
| Series: | Robotics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2218-6581/8/1/9 |
| _version_ | 1811308011476484096 |
|---|---|
| author | Abhilash Pandya Shahab Eslamian Hao Ying Matthew Nokleby Luke A. Reisner |
| author_facet | Abhilash Pandya Shahab Eslamian Hao Ying Matthew Nokleby Luke A. Reisner |
| author_sort | Abhilash Pandya |
| collection | DOAJ |
| description | This paper describes a recording and playback system developed using a da Vinci Standard Surgical System and research kit. The system records stereo laparoscopic videos, robot arm joint angles, and surgeon⁻console interactions in a synchronized manner. A user can then, on-demand and at adjustable speeds, watch stereo videos and feel recorded movements on the hand controllers of entire procedures or sub procedures. Currently, there is no reported comprehensive ability to capture expert surgeon movements and insights and reproduce them on hardware directly. This system has important applications in several areas: (1) training of surgeons, (2) collection of learning data for the development of advanced control algorithms and intelligent autonomous behaviors, and (3) use as a “black box„ for retrospective error analysis. We show a prototype of such an immersive system on a clinically-relevant platform along with its recording and playback fidelity. Lastly, we convey possible research avenues to create better systems for training and assisting robotic surgeons. |
| first_indexed | 2024-04-13T09:14:20Z |
| format | Article |
| id | doaj.art-619fcbfa54a34a94ace097671ad61fd6 |
| institution | Directory Open Access Journal |
| issn | 2218-6581 |
| language | English |
| last_indexed | 2024-04-13T09:14:20Z |
| publishDate | 2019-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Robotics |
| spelling | doaj.art-619fcbfa54a34a94ace097671ad61fd62022-12-22T02:52:46ZengMDPI AGRobotics2218-65812019-02-0181910.3390/robotics8010009robotics8010009A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical SystemAbhilash Pandya0Shahab Eslamian1Hao Ying2Matthew Nokleby3Luke A. Reisner4Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USAThis paper describes a recording and playback system developed using a da Vinci Standard Surgical System and research kit. The system records stereo laparoscopic videos, robot arm joint angles, and surgeon⁻console interactions in a synchronized manner. A user can then, on-demand and at adjustable speeds, watch stereo videos and feel recorded movements on the hand controllers of entire procedures or sub procedures. Currently, there is no reported comprehensive ability to capture expert surgeon movements and insights and reproduce them on hardware directly. This system has important applications in several areas: (1) training of surgeons, (2) collection of learning data for the development of advanced control algorithms and intelligent autonomous behaviors, and (3) use as a “black box„ for retrospective error analysis. We show a prototype of such an immersive system on a clinically-relevant platform along with its recording and playback fidelity. Lastly, we convey possible research avenues to create better systems for training and assisting robotic surgeons.https://www.mdpi.com/2218-6581/8/1/9robotic surgerysurgical trainingmachine learningsurgical automation |
| spellingShingle | Abhilash Pandya Shahab Eslamian Hao Ying Matthew Nokleby Luke A. Reisner A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System Robotics robotic surgery surgical training machine learning surgical automation |
| title | A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System |
| title_full | A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System |
| title_fullStr | A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System |
| title_full_unstemmed | A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System |
| title_short | A Robotic Recording and Playback Platform for Training Surgeons and Learning Autonomous Behaviors Using the da Vinci Surgical System |
| title_sort | robotic recording and playback platform for training surgeons and learning autonomous behaviors using the da vinci surgical system |
| topic | robotic surgery surgical training machine learning surgical automation |
| url | https://www.mdpi.com/2218-6581/8/1/9 |
| work_keys_str_mv | AT abhilashpandya aroboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT shahabeslamian aroboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT haoying aroboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT matthewnokleby aroboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT lukeareisner aroboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT abhilashpandya roboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT shahabeslamian roboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT haoying roboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT matthewnokleby roboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem AT lukeareisner roboticrecordingandplaybackplatformfortrainingsurgeonsandlearningautonomousbehaviorsusingthedavincisurgicalsystem |