Unifying Visual Localization and Scene Recognition for People With Visual Impairment

With the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?&#x201D...

Full description

Bibliographic Details
Main Authors: Ruiqi Cheng, Kaiwei Wang, Jian Bai, Zhijie Xu
Format: Article
Language:English
Published: IEEE 2020-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/9051719/
_version_ 1818428053002387456
author Ruiqi Cheng
Kaiwei Wang
Jian Bai
Zhijie Xu
author_facet Ruiqi Cheng
Kaiwei Wang
Jian Bai
Zhijie Xu
author_sort Ruiqi Cheng
collection DOAJ
description With the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?” are still to be resolved by taking advantage of visual information. In this paper, we leverage the prevailing compact network as the backbone to build a unified network featuring two branches that implement scene description and scene recognition separately. Based on the unified network, the proposed pipeline performs scene recognition and visual localization simultaneously in the scenario of assistive navigation. The visual localization pipeline involves image retrieval and sequence matching. In the experiments, different configurations of the proposed pipeline are tested on public datasets to search for the optimal parameters. Moreover, on the real-world datasets captured by the wearable assistive device, the proposed assistive navigation pipeline is proved to achieve satisfactory performance. On the challenging dataset, the top-5 precision of scene recognition is more than 80%, and the visual localization precision is over 60% under a recall of 60%. The related codes and datasets are open-source online at https://github.com/chengricky/ScenePlaceRecognition.
first_indexed 2024-12-14T14:55:30Z
format Article
id doaj.art-d32e856286c346a6a8093f4fb3ecd9df
institution Directory Open Access Journal
issn 2169-3536
language English
last_indexed 2024-12-14T14:55:30Z
publishDate 2020-01-01
publisher IEEE
record_format Article
series IEEE Access
spelling doaj.art-d32e856286c346a6a8093f4fb3ecd9df2022-12-21T22:57:00ZengIEEEIEEE Access2169-35362020-01-018642846429610.1109/ACCESS.2020.29847189051719Unifying Visual Localization and Scene Recognition for People With Visual ImpairmentRuiqi Cheng0https://orcid.org/0000-0001-7951-196XKaiwei Wang1Jian Bai2Zhijie Xu3https://orcid.org/0000-0002-0524-5926National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaNational Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaNational Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, ChinaSchool of Computing and Engineering, University of Huddersfield, Huddersfield, U.KWith the development of computer vision and mobile computing, assistive navigation for people with visual impairment arouses the attention of research communities. As two key challenges of assistive navigation, “Where am I?” and “What are the surroundings?” are still to be resolved by taking advantage of visual information. In this paper, we leverage the prevailing compact network as the backbone to build a unified network featuring two branches that implement scene description and scene recognition separately. Based on the unified network, the proposed pipeline performs scene recognition and visual localization simultaneously in the scenario of assistive navigation. The visual localization pipeline involves image retrieval and sequence matching. In the experiments, different configurations of the proposed pipeline are tested on public datasets to search for the optimal parameters. Moreover, on the real-world datasets captured by the wearable assistive device, the proposed assistive navigation pipeline is proved to achieve satisfactory performance. On the challenging dataset, the top-5 precision of scene recognition is more than 80%, and the visual localization precision is over 60% under a recall of 60%. The related codes and datasets are open-source online at https://github.com/chengricky/ScenePlaceRecognition.https://ieeexplore.ieee.org/document/9051719/Visual place recognitionglobal image descriptorscene classificationassistive navigation
spellingShingle Ruiqi Cheng
Kaiwei Wang
Jian Bai
Zhijie Xu
Unifying Visual Localization and Scene Recognition for People With Visual Impairment
IEEE Access
Visual place recognition
global image descriptor
scene classification
assistive navigation
title Unifying Visual Localization and Scene Recognition for People With Visual Impairment
title_full Unifying Visual Localization and Scene Recognition for People With Visual Impairment
title_fullStr Unifying Visual Localization and Scene Recognition for People With Visual Impairment
title_full_unstemmed Unifying Visual Localization and Scene Recognition for People With Visual Impairment
title_short Unifying Visual Localization and Scene Recognition for People With Visual Impairment
title_sort unifying visual localization and scene recognition for people with visual impairment
topic Visual place recognition
global image descriptor
scene classification
assistive navigation
url https://ieeexplore.ieee.org/document/9051719/
work_keys_str_mv AT ruiqicheng unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment
AT kaiweiwang unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment
AT jianbai unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment
AT zhijiexu unifyingvisuallocalizationandscenerecognitionforpeoplewithvisualimpairment