Patch-Based Difference-in-Level Detection with Segmented Ground Mask
Difference-in-level detection in outdoor scenes has various possible applications, including walking assistance for blind people, robot walking assistance, and mapping the hazards of factory premises. It is difficult to detect all outdoor differences in level, such as RGB or RGB-D images, not only i...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/4/806 |
| _version_ | 1797621353134686208 |
|---|---|
| author | Yusuke Nonaka Hideaki Uchiyama Hideo Saito Shoji Yachida Kota Iwamoto |
| author_facet | Yusuke Nonaka Hideaki Uchiyama Hideo Saito Shoji Yachida Kota Iwamoto |
| author_sort | Yusuke Nonaka |
| collection | DOAJ |
| description | Difference-in-level detection in outdoor scenes has various possible applications, including walking assistance for blind people, robot walking assistance, and mapping the hazards of factory premises. It is difficult to detect all outdoor differences in level, such as RGB or RGB-D images, not only including road curbs, which are often targeted for detection in automated driving, but also differences in level on factory premises and sidewalks, because the pattern of outdoor differences in level is abundant and complex. This paper proposes a novel method for detecting differences in level from RGB-D images with segmented ground masks. First, image patches of differences in level were extracted from outdoor images to create the dataset. The change in the normal vector of the contour part on the detected plane is used to generate image patches of the difference in level, but this method strongly depends on the accuracy of planar detection, and it detects only some differences in level. Then, we created the dataset, consisting of image patches and including the extracted differences in level. The dataset is used for training a deep learning model for detecting differences in level in outdoor images without limitations. In addition, because the purpose of this paper is to detect differences in level in outdoor walking areas, regions in the image other than the target areas were excluded by the segmented ground mask. For the performance evaluation, we implemented our algorithm using a modern smartphone with a high-performance depth camera. To evaluate the effectiveness of the proposed method, the results from various inputs, such as RGB, depth, grayscale, normal, and combinations of them, were qualitatively and quantitatively evaluated, and Blender was used to generate synthetic test images for a quantitative evaluation of the difference in level. We confirm that the suggested method successfully detects various types of differences in level in outdoor images, even in complex scenes. |
| first_indexed | 2024-03-11T08:54:45Z |
| format | Article |
| id | doaj.art-34d30a6c9c9b4b8ab597708bea01cc6c |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-11T08:54:45Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-34d30a6c9c9b4b8ab597708bea01cc6c2023-11-16T20:10:22ZengMDPI AGElectronics2079-92922023-02-0112480610.3390/electronics12040806Patch-Based Difference-in-Level Detection with Segmented Ground MaskYusuke Nonaka0Hideaki Uchiyama1Hideo Saito2Shoji Yachida3Kota Iwamoto4Graduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanGraduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanGraduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanVisual Intelligence Research Lab., NEC Corporation, Kawasaki 211-8666, JapanVisual Intelligence Research Lab., NEC Corporation, Kawasaki 211-8666, JapanDifference-in-level detection in outdoor scenes has various possible applications, including walking assistance for blind people, robot walking assistance, and mapping the hazards of factory premises. It is difficult to detect all outdoor differences in level, such as RGB or RGB-D images, not only including road curbs, which are often targeted for detection in automated driving, but also differences in level on factory premises and sidewalks, because the pattern of outdoor differences in level is abundant and complex. This paper proposes a novel method for detecting differences in level from RGB-D images with segmented ground masks. First, image patches of differences in level were extracted from outdoor images to create the dataset. The change in the normal vector of the contour part on the detected plane is used to generate image patches of the difference in level, but this method strongly depends on the accuracy of planar detection, and it detects only some differences in level. Then, we created the dataset, consisting of image patches and including the extracted differences in level. The dataset is used for training a deep learning model for detecting differences in level in outdoor images without limitations. In addition, because the purpose of this paper is to detect differences in level in outdoor walking areas, regions in the image other than the target areas were excluded by the segmented ground mask. For the performance evaluation, we implemented our algorithm using a modern smartphone with a high-performance depth camera. To evaluate the effectiveness of the proposed method, the results from various inputs, such as RGB, depth, grayscale, normal, and combinations of them, were qualitatively and quantitatively evaluated, and Blender was used to generate synthetic test images for a quantitative evaluation of the difference in level. We confirm that the suggested method successfully detects various types of differences in level in outdoor images, even in complex scenes.https://www.mdpi.com/2079-9292/12/4/806deference-in-level detectionsegmentationsupervised learningclassificationoutdoor navigationCNN network |
| spellingShingle | Yusuke Nonaka Hideaki Uchiyama Hideo Saito Shoji Yachida Kota Iwamoto Patch-Based Difference-in-Level Detection with Segmented Ground Mask Electronics deference-in-level detection segmentation supervised learning classification outdoor navigation CNN network |
| title | Patch-Based Difference-in-Level Detection with Segmented Ground Mask |
| title_full | Patch-Based Difference-in-Level Detection with Segmented Ground Mask |
| title_fullStr | Patch-Based Difference-in-Level Detection with Segmented Ground Mask |
| title_full_unstemmed | Patch-Based Difference-in-Level Detection with Segmented Ground Mask |
| title_short | Patch-Based Difference-in-Level Detection with Segmented Ground Mask |
| title_sort | patch based difference in level detection with segmented ground mask |
| topic | deference-in-level detection segmentation supervised learning classification outdoor navigation CNN network |
| url | https://www.mdpi.com/2079-9292/12/4/806 |
| work_keys_str_mv | AT yusukenonaka patchbaseddifferenceinleveldetectionwithsegmentedgroundmask AT hideakiuchiyama patchbaseddifferenceinleveldetectionwithsegmentedgroundmask AT hideosaito patchbaseddifferenceinleveldetectionwithsegmentedgroundmask AT shojiyachida patchbaseddifferenceinleveldetectionwithsegmentedgroundmask AT kotaiwamoto patchbaseddifferenceinleveldetectionwithsegmentedgroundmask |