Fast semantic-aware motion state detection for visual SLAM in dynamic environment
Existing visual SLAM (vSLAM) systems fail to perform well in dynamic environments as they cannot effectively ignore moving objects during pose estimation and mapping. We propose a lightweight approach to improve the robustness of existing feature based RGB-D and stereo vSLAM by accurately removing d...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/178580 |
| _version_ | 1811691220851752960 |
|---|---|
| author | Singh, Gaurav Wu, Meiqing Do, Minh Van Lam, Siew-Kei |
| author2 | College of Computing and Data Science |
| author_facet | College of Computing and Data Science Singh, Gaurav Wu, Meiqing Do, Minh Van Lam, Siew-Kei |
| author_sort | Singh, Gaurav |
| collection | NTU |
| description | Existing visual SLAM (vSLAM) systems fail to perform well in dynamic environments as they cannot effectively ignore moving objects during pose estimation and mapping. We propose a lightweight approach to improve the robustness of existing feature based RGB-D and stereo vSLAM by accurately removing dynamic outliers in the scene that contribute to failures in pose estimation and mapping. First, a novel motion state detection algorithm using the depth and feature flow information is presented to identify regions in the scene with high moving probability. This information is then fused with semantic cues via a probability framework to enable accurate and robust moving object extraction to retain the useful features for pose estimation and mapping. To reduce the computational complexity of extracting semantic information in every frame, we propose to extract semantics only on keyframes with significant changes in image content. Semantic propagation is used to compensate for the changes in the intermediate frames (i.e., non-keyframes). This is achieved by computing the dense transformation map using the available feature flow vectors. The proposed techniques can be integrated into existing vSLAM systems to increase their robustness in dynamic environments without incurring much computation cost. Our work highlights the importance of distinguishing between motion states of potential moving objects for vSLAM in highly dynamic environments. We provide extensive experimental results on four well-known RGB-D and stereo datasets to show that the proposed technique outperforms existing vSLAM methods in indoor and outdoor environments under various dynamic scenarios including crowded scenes. We also perform our experiments on a low-cost embedded platform, i.e., Jetson TX1, to demonstrate the computational efficiency of our method. |
| first_indexed | 2024-10-01T06:16:26Z |
| format | Journal Article |
| id | ntu-10356/178580 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:16:26Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1785802024-06-28T00:46:35Z Fast semantic-aware motion state detection for visual SLAM in dynamic environment Singh, Gaurav Wu, Meiqing Do, Minh Van Lam, Siew-Kei College of Computing and Data Science School of Computer Science and Engineering Computer and Information Science Visual SLAM Semantic segmentation Existing visual SLAM (vSLAM) systems fail to perform well in dynamic environments as they cannot effectively ignore moving objects during pose estimation and mapping. We propose a lightweight approach to improve the robustness of existing feature based RGB-D and stereo vSLAM by accurately removing dynamic outliers in the scene that contribute to failures in pose estimation and mapping. First, a novel motion state detection algorithm using the depth and feature flow information is presented to identify regions in the scene with high moving probability. This information is then fused with semantic cues via a probability framework to enable accurate and robust moving object extraction to retain the useful features for pose estimation and mapping. To reduce the computational complexity of extracting semantic information in every frame, we propose to extract semantics only on keyframes with significant changes in image content. Semantic propagation is used to compensate for the changes in the intermediate frames (i.e., non-keyframes). This is achieved by computing the dense transformation map using the available feature flow vectors. The proposed techniques can be integrated into existing vSLAM systems to increase their robustness in dynamic environments without incurring much computation cost. Our work highlights the importance of distinguishing between motion states of potential moving objects for vSLAM in highly dynamic environments. We provide extensive experimental results on four well-known RGB-D and stereo datasets to show that the proposed technique outperforms existing vSLAM methods in indoor and outdoor environments under various dynamic scenarios including crowded scenes. We also perform our experiments on a low-cost embedded platform, i.e., Jetson TX1, to demonstrate the computational efficiency of our method. Ministry of Education (MOE) Nanyang Technological University This work was supported in part by the RIE2020 Industry Alignment Fund—Industry Collaboration Projects (IAF-ICP) Funding Initiative; in part by the Singapore Telecommunications Ltd. (Singtel), through Singtel Cognitive and Artificial Intelligence Laboratory for Enterprises [SCALE@Nanyang Technological University (NTU)] (cash and in-kind contribution); and in part by the Ministry of Education, Singapore, under its Academic Research Fund Tier 1, under Grant RG78/21. 2024-06-28T00:46:35Z 2024-06-28T00:46:35Z 2022 Journal Article Singh, G., Wu, M., Do, M. V. & Lam, S. (2022). Fast semantic-aware motion state detection for visual SLAM in dynamic environment. IEEE Transactions On Intelligent Transportation Systems, 23(12), 23014-23030. https://dx.doi.org/10.1109/TITS.2022.3213694 1524-9050 https://hdl.handle.net/10356/178580 10.1109/TITS.2022.3213694 2-s2.0-85141467556 12 23 23014 23030 en RG78/21 IEEE Transactions on Intelligent Transportation Systems © 2022 IEEE. All rights reserved. |
| spellingShingle | Computer and Information Science Visual SLAM Semantic segmentation Singh, Gaurav Wu, Meiqing Do, Minh Van Lam, Siew-Kei Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title | Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title_full | Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title_fullStr | Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title_full_unstemmed | Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title_short | Fast semantic-aware motion state detection for visual SLAM in dynamic environment |
| title_sort | fast semantic aware motion state detection for visual slam in dynamic environment |
| topic | Computer and Information Science Visual SLAM Semantic segmentation |
| url | https://hdl.handle.net/10356/178580 |
| work_keys_str_mv | AT singhgaurav fastsemanticawaremotionstatedetectionforvisualslamindynamicenvironment AT wumeiqing fastsemanticawaremotionstatedetectionforvisualslamindynamicenvironment AT dominhvan fastsemanticawaremotionstatedetectionforvisualslamindynamicenvironment AT lamsiewkei fastsemanticawaremotionstatedetectionforvisualslamindynamicenvironment |