REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION
In this paper we present the semantic SLAM method based on a bundle of deep convolutional neural networks. It provides real-time dense semantic scene reconstruction for the autonomous driving system of an off-road robotic vehicle. Most state-of-the-art neural networks require large computing resourc...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2021-06-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2021/399/2021/isprs-archives-XLIII-B2-2021-399-2021.pdf |
| _version_ | 1819289887902269440 |
|---|---|
| author | B. Vishnyakov I. Sgibnev V. Sheverdin A. Sorokin P. Masalov K. Kazakhmedov S. Arseev |
| author_facet | B. Vishnyakov I. Sgibnev V. Sheverdin A. Sorokin P. Masalov K. Kazakhmedov S. Arseev |
| author_sort | B. Vishnyakov |
| collection | DOAJ |
| description | In this paper we present the semantic SLAM method based on a bundle of deep convolutional neural networks. It provides real-time dense semantic scene reconstruction for the autonomous driving system of an off-road robotic vehicle. Most state-of-the-art neural networks require large computing resources that go beyond the capabilities of many robotic platforms. We propose an architecture for 3D semantic scene reconstruction on top of the recent progress in computer vision by integrating SuperPoint, SuperGlue, Bi3D, DeepLabV3+, RTM3D and additional module with pre-processing, inference and postprocessing operations performed on GPU. We also updated our simulated dataset for semantic segmentation and added disparity images. |
| first_indexed | 2024-12-24T03:14:00Z |
| format | Article |
| id | doaj.art-746c873827564698bc0d76b232322c29 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-12-24T03:14:00Z |
| publishDate | 2021-06-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-746c873827564698bc0d76b232322c292022-12-21T17:17:42ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342021-06-01XLIII-B2-202139940410.5194/isprs-archives-XLIII-B2-2021-399-2021REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATIONB. Vishnyakov0I. Sgibnev1V. Sheverdin2A. Sorokin3P. Masalov4K. Kazakhmedov5S. Arseev6FGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaFGUP «State Research Institute of Aviation Systems», 7, Viktorenko Street, Moscow, 125319, RussiaIn this paper we present the semantic SLAM method based on a bundle of deep convolutional neural networks. It provides real-time dense semantic scene reconstruction for the autonomous driving system of an off-road robotic vehicle. Most state-of-the-art neural networks require large computing resources that go beyond the capabilities of many robotic platforms. We propose an architecture for 3D semantic scene reconstruction on top of the recent progress in computer vision by integrating SuperPoint, SuperGlue, Bi3D, DeepLabV3+, RTM3D and additional module with pre-processing, inference and postprocessing operations performed on GPU. We also updated our simulated dataset for semantic segmentation and added disparity images.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2021/399/2021/isprs-archives-XLIII-B2-2021-399-2021.pdf |
| spellingShingle | B. Vishnyakov I. Sgibnev V. Sheverdin A. Sorokin P. Masalov K. Kazakhmedov S. Arseev REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION |
| title_full | REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION |
| title_fullStr | REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION |
| title_full_unstemmed | REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION |
| title_short | REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION |
| title_sort | real time semantic slam with dcnn based feature point detection matching and dense point cloud aggregation |
| url | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2021/399/2021/isprs-archives-XLIII-B2-2021-399-2021.pdf |
| work_keys_str_mv | AT bvishnyakov realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT isgibnev realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT vsheverdin realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT asorokin realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT pmasalov realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT kkazakhmedov realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation AT sarseev realtimesemanticslamwithdcnnbasedfeaturepointdetectionmatchinganddensepointcloudaggregation |