Unifying Visual Localization and Scene Recognition for People With Visual Impairment
With the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?”...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9051719/ |
_version_ | 1818428053002387456 |
---|---|
author | Ruiqi Cheng Kaiwei Wang Jian Bai Zhijie Xu |
author_facet | Ruiqi Cheng Kaiwei Wang Jian Bai Zhijie Xu |
author_sort | Ruiqi Cheng |
collection | DOAJ |
description | With the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?” are still to be resolved by taking advantage of visual information. In this paper, we leverage the prevailing compact network as the backbone to build a unified network featuring two branches that implement scene description and scene recognition separately. Based on the unified network, the proposed pipeline performs scene recognition and visual localization simultaneously in the scenario of assistive navigation. The visual localization pipeline involves image retrieval and sequence matching. In the experiments, different configurations of the proposed pipeline are tested on public datasets to search for the optimal parameters. Moreover, on the real-world datasets captured by the wearable assistive device, the proposed assistive navigation pipeline is proved to achieve satisfactory performance. On the challenging dataset, the top-5 precision of scene recognition is more than 80%, and the visual localization precision is over 60% under a recall of 60%. The related codes and datasets are open-source online at https://github.com/chengricky/ScenePlaceRecognition. |
first_indexed | 2024-12-14T14:55:30Z |
format | Article |
id | doaj.art-d32e856286c346a6a8093f4fb3ecd9df |
institution | Directory Open Access Journal |
issn | 2169-3536 |
language | English |
last_indexed | 2024-12-14T14:55:30Z |
publishDate | 2020-01-01 |
publisher | IEEE |
record_format | Article |
series | IEEE Access |
spelling | doaj.art-d32e856286c346a6a8093f4fb3ecd9df2022-12-21T22:57:00ZengIEEEIEEE Access2169-35362020-01-018642846429610.1109/ACCESS.2020.29847189051719Unifying Visual Localization and Scene Recognition for People With Visual ImpairmentRuiqi Cheng0https://orcid.org/0000-0001-7951-196XKaiwei Wang1Jian Bai2Zhijie Xu3https://orcid.org/0000-0002-0524-5926National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaNational Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaNational Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaSchool of Computing and Engineering, University of Huddersfield, Huddersfield, U.KWith the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?” are still to be resolved by taking advantage of visual information. In this paper, we leverage the prevailing compact network as the backbone to build a unified network featuring two branches that implement scene description and scene recognition separately. Based on the unified network, the proposed pipeline performs scene recognition and visual localization simultaneously in the scenario of assistive navigation. The visual localization pipeline involves image retrieval and sequence matching. In the experiments, different configurations of the proposed pipeline are tested on public datasets to search for the optimal parameters. Moreover, on the real-world datasets captured by the wearable assistive device, the proposed assistive navigation pipeline is proved to achieve satisfactory performance. On the challenging dataset, the top-5 precision of scene recognition is more than 80%, and the visual localization precision is over 60% under a recall of 60%. The related codes and datasets are open-source online at https://github.com/chengricky/ScenePlaceRecognition.https://ieeexplore.ieee.org/document/9051719/Visual place recognitionglobal image descriptorscene classificationassistive navigation |
spellingShingle | Ruiqi Cheng Kaiwei Wang Jian Bai Zhijie Xu Unifying Visual Localization and Scene Recognition for People With Visual Impairment IEEE Access Visual place recognition global image descriptor scene classification assistive navigation |
title | Unifying Visual Localization and Scene Recognition for People With Visual Impairment |
title_full | Unifying Visual Localization and Scene Recognition for People With Visual Impairment |
title_fullStr | Unifying Visual Localization and Scene Recognition for People With Visual Impairment |
title_full_unstemmed | Unifying Visual Localization and Scene Recognition for People With Visual Impairment |
title_short | Unifying Visual Localization and Scene Recognition for People With Visual Impairment |
title_sort | unifying visual localization and scene recognition for people with visual impairment |
topic | Visual place recognition global image descriptor scene classification assistive navigation |
url | https://ieeexplore.ieee.org/document/9051719/ |
work_keys_str_mv | AT ruiqicheng unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment AT kaiweiwang unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment AT jianbai unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment AT zhijiexu unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment |