A visual questioning answering approach to enhance robot localization in indoor environments
Navigating robots with precision in complex environments remains a significant challenge. In this article, we present an innovative approach to enhance robot localization in dynamic and intricate spaces like homes and offices. We leverage Visual Question Answering (VQA) techniques to integrate seman...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-11-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1290584/full |
| _version_ | 1797450821965709312 |
|---|---|
| author | Juan Diego Peña-Narvaez Francisco Martín José Miguel Guerrero Rodrigo Pérez-Rodríguez |
| author_facet | Juan Diego Peña-Narvaez Francisco Martín José Miguel Guerrero Rodrigo Pérez-Rodríguez |
| author_sort | Juan Diego Peña-Narvaez |
| collection | DOAJ |
| description | Navigating robots with precision in complex environments remains a significant challenge. In this article, we present an innovative approach to enhance robot localization in dynamic and intricate spaces like homes and offices. We leverage Visual Question Answering (VQA) techniques to integrate semantic insights into traditional mapping methods, formulating a novel position hypothesis generation to assist localization methods, while also addressing challenges related to mapping accuracy and localization reliability. Our methodology combines a probabilistic approach with the latest advances in Monte Carlo Localization methods and Visual Language models. The integration of our hypothesis generation mechanism results in more robust robot localization compared to existing approaches. Experimental validation demonstrates the effectiveness of our approach, surpassing state-of-the-art multi-hypothesis algorithms in both position estimation and particle quality. This highlights the potential for accurate self-localization, even in symmetric environments with large corridor spaces. Furthermore, our approach exhibits a high recovery rate from deliberate position alterations, showcasing its robustness. By merging visual sensing, semantic mapping, and advanced localization techniques, we open new horizons for robot navigation. Our work bridges the gap between visual perception, semantic understanding, and traditional mapping, enabling robots to interact with their environment through questions and enrich their map with valuable insights. The code for this project is available on GitHub https://github.com/juandpenan/topology_nav_ros2. |
| first_indexed | 2024-03-09T14:46:10Z |
| format | Article |
| id | doaj.art-1d0f5b8ebb004a779d7034d0a59a75bc |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-03-09T14:46:10Z |
| publishDate | 2023-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-1d0f5b8ebb004a779d7034d0a59a75bc2023-11-27T06:17:32ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182023-11-011710.3389/fnbot.2023.12905841290584A visual questioning answering approach to enhance robot localization in indoor environmentsJuan Diego Peña-Narvaez0Francisco Martín1José Miguel Guerrero2Rodrigo Pérez-Rodríguez3Intelligent Robotics Lab, Signal Theory, Communications, Telematics Systems, and Computation Department, International Doctoral School, Rey Juan Carlos University, Fuenlabrada, SpainIntelligent Robotics Lab, Signal Theory, Communications, Telematics Systems, and Computation Department, Rey Juan Carlos University, Fuenlabrada, SpainIntelligent Robotics Lab, Signal Theory, Communications, Telematics Systems, and Computation Department, Rey Juan Carlos University, Fuenlabrada, SpainIntelligent Robotics Lab, Signal Theory, Communications, Telematics Systems, and Computation Department, Rey Juan Carlos University, Fuenlabrada, SpainNavigating robots with precision in complex environments remains a significant challenge. In this article, we present an innovative approach to enhance robot localization in dynamic and intricate spaces like homes and offices. We leverage Visual Question Answering (VQA) techniques to integrate semantic insights into traditional mapping methods, formulating a novel position hypothesis generation to assist localization methods, while also addressing challenges related to mapping accuracy and localization reliability. Our methodology combines a probabilistic approach with the latest advances in Monte Carlo Localization methods and Visual Language models. The integration of our hypothesis generation mechanism results in more robust robot localization compared to existing approaches. Experimental validation demonstrates the effectiveness of our approach, surpassing state-of-the-art multi-hypothesis algorithms in both position estimation and particle quality. This highlights the potential for accurate self-localization, even in symmetric environments with large corridor spaces. Furthermore, our approach exhibits a high recovery rate from deliberate position alterations, showcasing its robustness. By merging visual sensing, semantic mapping, and advanced localization techniques, we open new horizons for robot navigation. Our work bridges the gap between visual perception, semantic understanding, and traditional mapping, enabling robots to interact with their environment through questions and enrich their map with valuable insights. The code for this project is available on GitHub https://github.com/juandpenan/topology_nav_ros2.https://www.frontiersin.org/articles/10.3389/fnbot.2023.1290584/fullvisual question answeringrobot localizationrobot navigationsemantic maprobot mapping |
| spellingShingle | Juan Diego Peña-Narvaez Francisco Martín José Miguel Guerrero Rodrigo Pérez-Rodríguez A visual questioning answering approach to enhance robot localization in indoor environments Frontiers in Neurorobotics visual question answering robot localization robot navigation semantic map robot mapping |
| title | A visual questioning answering approach to enhance robot localization in indoor environments |
| title_full | A visual questioning answering approach to enhance robot localization in indoor environments |
| title_fullStr | A visual questioning answering approach to enhance robot localization in indoor environments |
| title_full_unstemmed | A visual questioning answering approach to enhance robot localization in indoor environments |
| title_short | A visual questioning answering approach to enhance robot localization in indoor environments |
| title_sort | visual questioning answering approach to enhance robot localization in indoor environments |
| topic | visual question answering robot localization robot navigation semantic map robot mapping |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1290584/full |
| work_keys_str_mv | AT juandiegopenanarvaez avisualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT franciscomartin avisualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT josemiguelguerrero avisualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT rodrigoperezrodriguez avisualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT juandiegopenanarvaez visualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT franciscomartin visualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT josemiguelguerrero visualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments AT rodrigoperezrodriguez visualquestioningansweringapproachtoenhancerobotlocalizationinindoorenvironments |