Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery
Abstract Accurate 3D shape measurement is crucial for surgical support and alignment in robotic surgery systems. Stereo cameras in laparoscopes offer a potential solution; however, their accuracy in stereo image matching diminishes when the target image has few textures. Although stereo matching wit...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-04-01
|
| Series: | Healthcare Technology Letters |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/htl2.12083 |
| _version_ | 1797202609858150400 |
|---|---|
| author | Ryo Furukawa Elvis Chen Ryusuke Sagawa Shiro Oka Hiroshi Kawasaki |
| author_facet | Ryo Furukawa Elvis Chen Ryusuke Sagawa Shiro Oka Hiroshi Kawasaki |
| author_sort | Ryo Furukawa |
| collection | DOAJ |
| description | Abstract Accurate 3D shape measurement is crucial for surgical support and alignment in robotic surgery systems. Stereo cameras in laparoscopes offer a potential solution; however, their accuracy in stereo image matching diminishes when the target image has few textures. Although stereo matching with deep learning has gained significant attention, supervised learning requires a large dataset of images with depth annotations, which are scarce for laparoscopes. Thus, there is a strong demand to explore alternative methods for depth reconstruction or annotation for laparoscopes. Active stereo techniques are a promising approach for achieving 3D reconstruction without textures. In this study, a 3D shape reconstruction method is proposed using an ultra‐small patterned projector attached to a laparoscopic arm to address these issues. The pattern projector emits a structured light with a grid‐like pattern that features node‐wise modulation for positional encoding. To scan the target object, multiple images are taken while the projector is in motion, and the relative poses of the projector and a camera are auto‐calibrated using a differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using images obtained from a surgical robot and comparing the results with a ground‐truth shape obtained from X‐ray CT. |
| first_indexed | 2024-04-24T08:06:10Z |
| format | Article |
| id | doaj.art-7dda74cac2104f4c84f747f7fcf05168 |
| institution | Directory Open Access Journal |
| issn | 2053-3713 |
| language | English |
| last_indexed | 2024-04-24T08:06:10Z |
| publishDate | 2024-04-01 |
| publisher | Wiley |
| record_format | Article |
| series | Healthcare Technology Letters |
| spelling | doaj.art-7dda74cac2104f4c84f747f7fcf051682024-04-17T10:06:56ZengWileyHealthcare Technology Letters2053-37132024-04-01112-319620510.1049/htl2.12083Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgeryRyo Furukawa0Elvis Chen1Ryusuke Sagawa2Shiro Oka3Hiroshi Kawasaki4Department of Informatics Kindai University Higashihiroshima JapanRobarts Research Institute London CanadaArtificial Intelligence Research Center National Institute of Anvanced Industrial Science and Technology (AIST) Tsukuba JapanHiroshima University Hiroshima JapanFaculty of Information Science and Electrical Engineering Kyushu University Fukuoka JapanAbstract Accurate 3D shape measurement is crucial for surgical support and alignment in robotic surgery systems. Stereo cameras in laparoscopes offer a potential solution; however, their accuracy in stereo image matching diminishes when the target image has few textures. Although stereo matching with deep learning has gained significant attention, supervised learning requires a large dataset of images with depth annotations, which are scarce for laparoscopes. Thus, there is a strong demand to explore alternative methods for depth reconstruction or annotation for laparoscopes. Active stereo techniques are a promising approach for achieving 3D reconstruction without textures. In this study, a 3D shape reconstruction method is proposed using an ultra‐small patterned projector attached to a laparoscopic arm to address these issues. The pattern projector emits a structured light with a grid‐like pattern that features node‐wise modulation for positional encoding. To scan the target object, multiple images are taken while the projector is in motion, and the relative poses of the projector and a camera are auto‐calibrated using a differential rendering technique. In the experiment, the proposed method is evaluated by performing 3D reconstruction using images obtained from a surgical robot and comparing the results with a ground‐truth shape obtained from X‐ray CT.https://doi.org/10.1049/htl2.12083biomedical imagingbiomedical measurementendoscopes |
| spellingShingle | Ryo Furukawa Elvis Chen Ryusuke Sagawa Shiro Oka Hiroshi Kawasaki Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery Healthcare Technology Letters biomedical imaging biomedical measurement endoscopes |
| title | Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery |
| title_full | Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery |
| title_fullStr | Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery |
| title_full_unstemmed | Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery |
| title_short | Calibration‐free structured‐light‐based 3D scanning system in laparoscope for robotic surgery |
| title_sort | calibration free structured light based 3d scanning system in laparoscope for robotic surgery |
| topic | biomedical imaging biomedical measurement endoscopes |
| url | https://doi.org/10.1049/htl2.12083 |
| work_keys_str_mv | AT ryofurukawa calibrationfreestructuredlightbased3dscanningsysteminlaparoscopeforroboticsurgery AT elvischen calibrationfreestructuredlightbased3dscanningsysteminlaparoscopeforroboticsurgery AT ryusukesagawa calibrationfreestructuredlightbased3dscanningsysteminlaparoscopeforroboticsurgery AT shirooka calibrationfreestructuredlightbased3dscanningsysteminlaparoscopeforroboticsurgery AT hiroshikawasaki calibrationfreestructuredlightbased3dscanningsysteminlaparoscopeforroboticsurgery |