A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot
Positioning a camera during laparoscopic and robotic procedures is challenging and essential for successful operations. During surgery, if the camera view is not optimal, surgery becomes more complex and potentially error-prone. To address this need, we have developed a voice interface to an autonom...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Robotics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2218-6581/11/2/40 |
| _version_ | 1797443910466797568 |
|---|---|
| author | Maysara Elazzazi Luay Jawad Mohammed Hilfi Abhilash Pandya |
| author_facet | Maysara Elazzazi Luay Jawad Mohammed Hilfi Abhilash Pandya |
| author_sort | Maysara Elazzazi |
| collection | DOAJ |
| description | Positioning a camera during laparoscopic and robotic procedures is challenging and essential for successful operations. During surgery, if the camera view is not optimal, surgery becomes more complex and potentially error-prone. To address this need, we have developed a voice interface to an autonomous camera system that can trigger behavioral changes and be more of a partner to the surgeon. Similarly to a human operator, the camera can take cues from the surgeon to help create optimized surgical camera views. It has the advantage of nominal behavior that is helpful in most general cases and has a natural language interface that makes it dynamically customizable and on-demand. It permits the control of a camera with a higher level of abstraction. This paper shows the implementation details and usability of a voice-activated autonomous camera system. A voice activation test on a limited set of practiced key phrases was performed using both online and offline voice recognition systems. The results show an on-average greater than 94% recognition accuracy for the online system and 86% accuracy for the offline system. However, the response time of the online system was greater than 1.5 s, whereas the local system was 0.6 s. This work is a step towards cooperative surgical robots that will effectively partner with human operators to enable more robust surgeries. A video link of the system in operation is provided in this paper. |
| first_indexed | 2024-03-09T13:03:56Z |
| format | Article |
| id | doaj.art-8ba35d4dffe54cbfb560b0631af9eb80 |
| institution | Directory Open Access Journal |
| issn | 2218-6581 |
| language | English |
| last_indexed | 2024-03-09T13:03:56Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Robotics |
| spelling | doaj.art-8ba35d4dffe54cbfb560b0631af9eb802023-11-30T21:51:14ZengMDPI AGRobotics2218-65812022-03-011124010.3390/robotics11020040A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical RobotMaysara Elazzazi0Luay Jawad1Mohammed Hilfi2Abhilash Pandya3Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Computer Science, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USADepartment of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USAPositioning a camera during laparoscopic and robotic procedures is challenging and essential for successful operations. During surgery, if the camera view is not optimal, surgery becomes more complex and potentially error-prone. To address this need, we have developed a voice interface to an autonomous camera system that can trigger behavioral changes and be more of a partner to the surgeon. Similarly to a human operator, the camera can take cues from the surgeon to help create optimized surgical camera views. It has the advantage of nominal behavior that is helpful in most general cases and has a natural language interface that makes it dynamically customizable and on-demand. It permits the control of a camera with a higher level of abstraction. This paper shows the implementation details and usability of a voice-activated autonomous camera system. A voice activation test on a limited set of practiced key phrases was performed using both online and offline voice recognition systems. The results show an on-average greater than 94% recognition accuracy for the online system and 86% accuracy for the offline system. However, the response time of the online system was greater than 1.5 s, whereas the local system was 0.6 s. This work is a step towards cooperative surgical robots that will effectively partner with human operators to enable more robust surgeries. A video link of the system in operation is provided in this paper.https://www.mdpi.com/2218-6581/11/2/40da Vincirobotic surgerylaparoscopic surgeryautonomous camera controlnatural language processing |
| spellingShingle | Maysara Elazzazi Luay Jawad Mohammed Hilfi Abhilash Pandya A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot Robotics da Vinci robotic surgery laparoscopic surgery autonomous camera control natural language processing |
| title | A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot |
| title_full | A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot |
| title_fullStr | A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot |
| title_full_unstemmed | A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot |
| title_short | A Natural Language Interface for an Autonomous Camera Control System on the da Vinci Surgical Robot |
| title_sort | natural language interface for an autonomous camera control system on the da vinci surgical robot |
| topic | da Vinci robotic surgery laparoscopic surgery autonomous camera control natural language processing |
| url | https://www.mdpi.com/2218-6581/11/2/40 |
| work_keys_str_mv | AT maysaraelazzazi anaturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT luayjawad anaturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT mohammedhilfi anaturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT abhilashpandya anaturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT maysaraelazzazi naturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT luayjawad naturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT mohammedhilfi naturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot AT abhilashpandya naturallanguageinterfaceforanautonomouscameracontrolsystemonthedavincisurgicalrobot |