A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice
Blind spot confirmation as a major challenge in surgical practice has led to booming of 3D modeling and printing technology and applications, but existing technologies could not offer readily available service of medical images in the operation room (OR) for surgeons. Contacting with keyboards and m...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8864002/ |
| _version_ | 1818433523335299072 |
|---|---|
| author | Wang Shi Hui Weihong Huang Jianzhong Hu Kun Tao Yonghong Peng |
| author_facet | Wang Shi Hui Weihong Huang Jianzhong Hu Kun Tao Yonghong Peng |
| author_sort | Wang Shi Hui |
| collection | DOAJ |
| description | Blind spot confirmation as a major challenge in surgical practice has led to booming of 3D modeling and printing technology and applications, but existing technologies could not offer readily available service of medical images in the operation room (OR) for surgeons. Contacting with keyboards and mice or asking assistants to help in viewing medical images during the operation could increase operation risk, infection and prolong of the operation time. This paper presents a new precise contactless medical image multimodal interaction system for surgeons in the OR. By supporting seamless multimodal interaction via 2D laser positioning, 3D gesture and voice recognition, it overcomes the design flaws in other similar solutions such as big motion for gesture recognition, round trip towards the screen of surgons, difficulty in selection and comparison of images, and more. Following the system engineering and evidence-based approaches, the design and development of this new precise contactless multimodal medical image interaction system meet surgeons' real world requirements for efficient and effective 2D medical images and 3D model operation under both preoperative and intraoperative scenarios. The system evaluation study is carried out in conjunction with a group of 10 neurosurgeons with 3 to 10 years' of surgical experience from different hospitals. Evaluation results successfully demonstrate the system's feasibility, efficiency and usability of in surgical contexts. |
| first_indexed | 2024-12-14T16:22:27Z |
| format | Article |
| id | doaj.art-99ec4f0581894bd99885d9bf85fb502d |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T16:22:27Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-99ec4f0581894bd99885d9bf85fb502d2022-12-21T22:54:46ZengIEEEIEEE Access2169-35362020-01-01812181112182010.1109/ACCESS.2019.29464048864002A New Precise Contactless Medical Image Multimodal Interaction System for Surgical PracticeWang Shi Hui0https://orcid.org/0000-0002-6856-8802Weihong Huang1https://orcid.org/0000-0002-7168-4943Jianzhong Hu2Kun Tao3Yonghong Peng4https://orcid.org/0000-0002-5508-1819Mobile Health Ministry of Education–China Mobile Joint Laboratory, Xiangya Hospital Central South University, Changsha, ChinaMobile Health Ministry of Education–China Mobile Joint Laboratory, Xiangya Hospital Central South University, Changsha, ChinaMobile Health Ministry of Education–China Mobile Joint Laboratory, Xiangya Hospital Central South University, Changsha, ChinaAIHealthX Company Ltd., Beijing, ChinaFaculty of Computer Science, University of Sunderland, Sunderland, U.K.Blind spot confirmation as a major challenge in surgical practice has led to booming of 3D modeling and printing technology and applications, but existing technologies could not offer readily available service of medical images in the operation room (OR) for surgeons. Contacting with keyboards and mice or asking assistants to help in viewing medical images during the operation could increase operation risk, infection and prolong of the operation time. This paper presents a new precise contactless medical image multimodal interaction system for surgeons in the OR. By supporting seamless multimodal interaction via 2D laser positioning, 3D gesture and voice recognition, it overcomes the design flaws in other similar solutions such as big motion for gesture recognition, round trip towards the screen of surgons, difficulty in selection and comparison of images, and more. Following the system engineering and evidence-based approaches, the design and development of this new precise contactless multimodal medical image interaction system meet surgeons' real world requirements for efficient and effective 2D medical images and 3D model operation under both preoperative and intraoperative scenarios. The system evaluation study is carried out in conjunction with a group of 10 neurosurgeons with 3 to 10 years' of surgical experience from different hospitals. Evaluation results successfully demonstrate the system's feasibility, efficiency and usability of in surgical contexts.https://ieeexplore.ieee.org/document/8864002/Contactless multimodal medical image systemhuman-computer interactionusabilityoperation context |
| spellingShingle | Wang Shi Hui Weihong Huang Jianzhong Hu Kun Tao Yonghong Peng A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice IEEE Access Contactless multimodal medical image system human-computer interaction usability operation context |
| title | A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice |
| title_full | A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice |
| title_fullStr | A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice |
| title_full_unstemmed | A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice |
| title_short | A New Precise Contactless Medical Image Multimodal Interaction System for Surgical Practice |
| title_sort | new precise contactless medical image multimodal interaction system for surgical practice |
| topic | Contactless multimodal medical image system human-computer interaction usability operation context |
| url | https://ieeexplore.ieee.org/document/8864002/ |
| work_keys_str_mv | AT wangshihui anewprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT weihonghuang anewprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT jianzhonghu anewprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT kuntao anewprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT yonghongpeng anewprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT wangshihui newprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT weihonghuang newprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT jianzhonghu newprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT kuntao newprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice AT yonghongpeng newprecisecontactlessmedicalimagemultimodalinteractionsystemforsurgicalpractice |