A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization
This paper proposes a novel multi-sensor-based indoor global localization system integrating visual localization aided by CNN-based image retrieval with a probabilistic localization approach. The global localization system consists of three parts: coarse place recognition, fine localization and re-l...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/19/2/249 |
| _version_ | 1818040587811553280 |
|---|---|
| author | Song Xu Wusheng Chou Hongyi Dong |
| author_facet | Song Xu Wusheng Chou Hongyi Dong |
| author_sort | Song Xu |
| collection | DOAJ |
| description | This paper proposes a novel multi-sensor-based indoor global localization system integrating visual localization aided by CNN-based image retrieval with a probabilistic localization approach. The global localization system consists of three parts: coarse place recognition, fine localization and re-localization from kidnapping. Coarse place recognition exploits a monocular camera to realize the initial localization based on image retrieval, in which off-the-shelf features extracted from a pre-trained Convolutional Neural Network (CNN) are adopted to determine the candidate locations of the robot. In the fine localization, a laser range finder is equipped to estimate the accurate pose of a mobile robot by means of an adaptive Monte Carlo localization, in which the candidate locations obtained by image retrieval are considered as seeds for initial random sampling. Additionally, to address the problem of robot kidnapping, we present a closed-loop localization mechanism to monitor the state of the robot in real time and make adaptive adjustments when the robot is kidnapped. The closed-loop mechanism effectively exploits the correlation of image sequences to realize the re-localization based on Long-Short Term Memory (LSTM) network. Extensive experiments were conducted and the results indicate that the proposed method not only exhibits great improvement on accuracy and speed, but also can recover from localization failures compared to two conventional localization methods. |
| first_indexed | 2024-12-10T08:16:54Z |
| format | Article |
| id | doaj.art-376c583ca9c14d5fbfcf5678da08904d |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-12-10T08:16:54Z |
| publishDate | 2019-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-376c583ca9c14d5fbfcf5678da08904d2022-12-22T01:56:26ZengMDPI AGSensors1424-82202019-01-0119224910.3390/s19020249s19020249A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo LocalizationSong Xu0Wusheng Chou1Hongyi Dong2School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, ChinaSchool of Mechanical Engineering and Automation, Beihang University, Beijing 100191, ChinaSchool of Mechanical Engineering and Automation, Beihang University, Beijing 100191, ChinaThis paper proposes a novel multi-sensor-based indoor global localization system integrating visual localization aided by CNN-based image retrieval with a probabilistic localization approach. The global localization system consists of three parts: coarse place recognition, fine localization and re-localization from kidnapping. Coarse place recognition exploits a monocular camera to realize the initial localization based on image retrieval, in which off-the-shelf features extracted from a pre-trained Convolutional Neural Network (CNN) are adopted to determine the candidate locations of the robot. In the fine localization, a laser range finder is equipped to estimate the accurate pose of a mobile robot by means of an adaptive Monte Carlo localization, in which the candidate locations obtained by image retrieval are considered as seeds for initial random sampling. Additionally, to address the problem of robot kidnapping, we present a closed-loop localization mechanism to monitor the state of the robot in real time and make adaptive adjustments when the robot is kidnapped. The closed-loop mechanism effectively exploits the correlation of image sequences to realize the re-localization based on Long-Short Term Memory (LSTM) network. Extensive experiments were conducted and the results indicate that the proposed method not only exhibits great improvement on accuracy and speed, but also can recover from localization failures compared to two conventional localization methods.http://www.mdpi.com/1424-8220/19/2/249indoor global localizationmonocular cameralaser range finderimage retrievalconvolutional neural networkkidnapping |
| spellingShingle | Song Xu Wusheng Chou Hongyi Dong A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization Sensors indoor global localization monocular camera laser range finder image retrieval convolutional neural network kidnapping |
| title | A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization |
| title_full | A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization |
| title_fullStr | A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization |
| title_full_unstemmed | A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization |
| title_short | A Robust Indoor Localization System Integrating Visual Localization Aided by CNN-Based Image Retrieval with Monte Carlo Localization |
| title_sort | robust indoor localization system integrating visual localization aided by cnn based image retrieval with monte carlo localization |
| topic | indoor global localization monocular camera laser range finder image retrieval convolutional neural network kidnapping |
| url | http://www.mdpi.com/1424-8220/19/2/249 |
| work_keys_str_mv | AT songxu arobustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization AT wushengchou arobustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization AT hongyidong arobustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization AT songxu robustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization AT wushengchou robustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization AT hongyidong robustindoorlocalizationsystemintegratingvisuallocalizationaidedbycnnbasedimageretrievalwithmontecarlolocalization |