Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation
Simultaneous localization and mapping (SLAM) is an important function for service robots to self-navigate modernized buildings. However, only a few existing applications allow them to automatically move between stories through elevator. Some approaches have accomplished with the aid of hardware; how...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8926334/ |
| _version_ | 1829493871110782976 |
|---|---|
| author | Shenlu Jiang Wei Yao Man-Sing Wong Meng Hang Zhonghua Hong Eun-Jin Kim Sung-Hyeon Joo Tae-Yong Kuc |
| author_facet | Shenlu Jiang Wei Yao Man-Sing Wong Meng Hang Zhonghua Hong Eun-Jin Kim Sung-Hyeon Joo Tae-Yong Kuc |
| author_sort | Shenlu Jiang |
| collection | DOAJ |
| description | Simultaneous localization and mapping (SLAM) is an important function for service robots to self-navigate modernized buildings. However, only a few existing applications allow them to automatically move between stories through elevator. Some approaches have accomplished with the aid of hardware; however, this study shows that computer vision can be a promising alternative for button localization. In this paper, we proposed a real-time multi-story SLAM system which overcomes the problem of detecting elevator buttons using a localization framework that combines tracking and detecting approaches. A two-stage deep neural network initially locates the original positions of the target buttons, and a part-based tracker follows the target buttons in real-time. A positive-negative classifier and deep learning neural network (particular for button shape detection) modify the tracker's output in every frame. To allow the robot to self-navigate, a 2D grid mapping approach was used for the localization and mapping. Then, when the robot navigates a floor, the A* algorithm generates the shortest path. In the experiment, two dynamic scenes (which include common elevator button localization challenges) were used to evaluate the efficiency of our approach, and compared it with other state-of-the-art methods. Our approach was also tested on a prototype robot system to assesses how well it can navigate a multi-story building. The results show that our method could overcome the common background challenges that occur inside an elevator, and in doing so, it enables the mobile robot to autonomously navigate a multi-story building. |
| first_indexed | 2024-12-16T06:42:18Z |
| format | Article |
| id | doaj.art-51ba65f200b447f6ae86f3255b760e05 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T06:42:18Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-51ba65f200b447f6ae86f3255b760e052022-12-21T22:40:39ZengIEEEIEEE Access2169-35362020-01-0181118113410.1109/ACCESS.2019.29580928926334Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story NavigationShenlu Jiang0https://orcid.org/0000-0001-6208-2142Wei Yao1https://orcid.org/0000-0001-7704-0615Man-Sing Wong2https://orcid.org/0000-0002-6439-6775Meng Hang3https://orcid.org/0000-0002-9290-1556Zhonghua Hong4https://orcid.org/0000-0003-0045-1066Eun-Jin Kim5https://orcid.org/0000-0002-4157-5316Sung-Hyeon Joo6https://orcid.org/0000-0001-7068-2324Tae-Yong Kuc7https://orcid.org/0000-0002-5816-0088College of Information and Communication Engineering, Sungkyunkwan University, Suwon, South KoreaDepartment of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong KongDepartment of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong KongSchool of Mechanical Engineering and Automation, Beihang University, Beijing, ChinaCollege of Information Technology, Shanghai Ocean University, Shanghai, ChinaCollege of Information and Communication Engineering, Sungkyunkwan University, Suwon, South KoreaCollege of Information and Communication Engineering, Sungkyunkwan University, Suwon, South KoreaCollege of Information and Communication Engineering, Sungkyunkwan University, Suwon, South KoreaSimultaneous localization and mapping (SLAM) is an important function for service robots to self-navigate modernized buildings. However, only a few existing applications allow them to automatically move between stories through elevator. Some approaches have accomplished with the aid of hardware; however, this study shows that computer vision can be a promising alternative for button localization. In this paper, we proposed a real-time multi-story SLAM system which overcomes the problem of detecting elevator buttons using a localization framework that combines tracking and detecting approaches. A two-stage deep neural network initially locates the original positions of the target buttons, and a part-based tracker follows the target buttons in real-time. A positive-negative classifier and deep learning neural network (particular for button shape detection) modify the tracker's output in every frame. To allow the robot to self-navigate, a 2D grid mapping approach was used for the localization and mapping. Then, when the robot navigates a floor, the A* algorithm generates the shortest path. In the experiment, two dynamic scenes (which include common elevator button localization challenges) were used to evaluate the efficiency of our approach, and compared it with other state-of-the-art methods. Our approach was also tested on a prototype robot system to assesses how well it can navigate a multi-story building. The results show that our method could overcome the common background challenges that occur inside an elevator, and in doing so, it enables the mobile robot to autonomously navigate a multi-story building.https://ieeexplore.ieee.org/document/8926334/Elevator button localizationmulti-story navigationobject detectionvisual trackingdeep learning |
| spellingShingle | Shenlu Jiang Wei Yao Man-Sing Wong Meng Hang Zhonghua Hong Eun-Jin Kim Sung-Hyeon Joo Tae-Yong Kuc Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation IEEE Access Elevator button localization multi-story navigation object detection visual tracking deep learning |
| title | Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation |
| title_full | Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation |
| title_fullStr | Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation |
| title_full_unstemmed | Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation |
| title_short | Automatic Elevator Button Localization Using a Combined Detecting and Tracking Framework for Multi-Story Navigation |
| title_sort | automatic elevator button localization using a combined detecting and tracking framework for multi story navigation |
| topic | Elevator button localization multi-story navigation object detection visual tracking deep learning |
| url | https://ieeexplore.ieee.org/document/8926334/ |
| work_keys_str_mv | AT shenlujiang automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT weiyao automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT mansingwong automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT menghang automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT zhonghuahong automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT eunjinkim automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT sunghyeonjoo automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation AT taeyongkuc automaticelevatorbuttonlocalizationusingacombineddetectingandtrackingframeworkformultistorynavigation |