Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy
Depth mapping from binocular endoscopy images plays an important role in stereoscopic surgical treatment. Owing to the development of deep convolutional neural networks (CNNs), binocular depth estimation models have achieved many exciting results in the fields of autonomous driving and machine visio...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9064889/ |
| _version_ | 1818444224938377216 |
|---|---|
| author | Xiong-Zhi Wang Yunfeng Nie Shao-Ping Lu Jingang Zhang |
| author_facet | Xiong-Zhi Wang Yunfeng Nie Shao-Ping Lu Jingang Zhang |
| author_sort | Xiong-Zhi Wang |
| collection | DOAJ |
| description | Depth mapping from binocular endoscopy images plays an important role in stereoscopic surgical treatment. Owing to the development of deep convolutional neural networks (CNNs), binocular depth estimation models have achieved many exciting results in the fields of autonomous driving and machine vision. However, the application of these methods to endoscopic imaging is greatly limited by the fact that binocular endoscopic images not only are rare, but also have unsatisfying features such as no texture, no ground truth, bad contrast, and high gloss. Aiming at solving the above-mentioned problems, we have built a precise gastrointestinal environment by the open-source software blender to simulate abundant binocular endoscopy data and proposed a 23-layer deep CNNs method to generate real-time stereo depth mapping. An efficient scale-invariant loss function is introduced in this paper to accommodate the characteristics of endoscope images, which improves the accuracy of achieved depth mapping results. Regarding the considerable training data for typical CNNs, our method requires only a few images ($960\times 720$ resolution) at 45 frames per second on an NVIDIA GTX 1080 GPU module, then the depth mapping information is generated in real-time with satisfactory accuracy. The effectiveness of the developed method is validated by comparing with state-of-the-art methods on processing the same datasets, demonstrating a faster and more accurate performance than other model frames. |
| first_indexed | 2024-12-14T19:12:33Z |
| format | Article |
| id | doaj.art-6a8b0d2d0ef741519cd598de328137e3 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T19:12:33Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-6a8b0d2d0ef741519cd598de328137e32022-12-21T22:50:41ZengIEEEIEEE Access2169-35362020-01-018732417324910.1109/ACCESS.2020.29877679064889Deep Convolutional Network for Stereo Depth Mapping in Binocular EndoscopyXiong-Zhi Wang0https://orcid.org/0000-0003-2601-5238Yunfeng Nie1Shao-Ping Lu2Jingang Zhang3School of Future Technology, University of Chinese Academy of Sciences, Beijing, ChinaDepartment of Applied Physics and Photoncis, Brussel Photonics, Vrije Universiteit Brussel, Brussels, BelgiumTKLNDST, CS, Nankai University, Tianjin, ChinaSchool of Future Technology, University of Chinese Academy of Sciences, Beijing, ChinaDepth mapping from binocular endoscopy images plays an important role in stereoscopic surgical treatment. Owing to the development of deep convolutional neural networks (CNNs), binocular depth estimation models have achieved many exciting results in the fields of autonomous driving and machine vision. However, the application of these methods to endoscopic imaging is greatly limited by the fact that binocular endoscopic images not only are rare, but also have unsatisfying features such as no texture, no ground truth, bad contrast, and high gloss. Aiming at solving the above-mentioned problems, we have built a precise gastrointestinal environment by the open-source software blender to simulate abundant binocular endoscopy data and proposed a 23-layer deep CNNs method to generate real-time stereo depth mapping. An efficient scale-invariant loss function is introduced in this paper to accommodate the characteristics of endoscope images, which improves the accuracy of achieved depth mapping results. Regarding the considerable training data for typical CNNs, our method requires only a few images ($960\times 720$ resolution) at 45 frames per second on an NVIDIA GTX 1080 GPU module, then the depth mapping information is generated in real-time with satisfactory accuracy. The effectiveness of the developed method is validated by comparing with state-of-the-art methods on processing the same datasets, demonstrating a faster and more accurate performance than other model frames.https://ieeexplore.ieee.org/document/9064889/Binocular endoscopesdeep convolutional neutral networkreal-time evaluationstereo depth mapping |
| spellingShingle | Xiong-Zhi Wang Yunfeng Nie Shao-Ping Lu Jingang Zhang Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy IEEE Access Binocular endoscopes deep convolutional neutral network real-time evaluation stereo depth mapping |
| title | Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy |
| title_full | Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy |
| title_fullStr | Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy |
| title_full_unstemmed | Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy |
| title_short | Deep Convolutional Network for Stereo Depth Mapping in Binocular Endoscopy |
| title_sort | deep convolutional network for stereo depth mapping in binocular endoscopy |
| topic | Binocular endoscopes deep convolutional neutral network real-time evaluation stereo depth mapping |
| url | https://ieeexplore.ieee.org/document/9064889/ |
| work_keys_str_mv | AT xiongzhiwang deepconvolutionalnetworkforstereodepthmappinginbinocularendoscopy AT yunfengnie deepconvolutionalnetworkforstereodepthmappinginbinocularendoscopy AT shaopinglu deepconvolutionalnetworkforstereodepthmappinginbinocularendoscopy AT jingangzhang deepconvolutionalnetworkforstereodepthmappinginbinocularendoscopy |