Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph
In this paper, we address the problem of autonomous exploration in unknown environments for ground mobile robots with deep reinforcement learning (DRL). To effectively explore unknown environments, we construct an exploration graph considering historical trajectories, frontier waypoints, landmarks,...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/18/8299 |
| _version_ | 1797520386928148480 |
|---|---|
| author | Zhiwen Zhang Chenghao Shi Pengming Zhu Zhiwen Zeng Hui Zhang |
| author_facet | Zhiwen Zhang Chenghao Shi Pengming Zhu Zhiwen Zeng Hui Zhang |
| author_sort | Zhiwen Zhang |
| collection | DOAJ |
| description | In this paper, we address the problem of autonomous exploration in unknown environments for ground mobile robots with deep reinforcement learning (DRL). To effectively explore unknown environments, we construct an exploration graph considering historical trajectories, frontier waypoints, landmarks, and obstacles. Meanwhile, to take full advantage of the spatiotemporal feature and historical information in the autonomous exploration task, we propose a novel network called Spatiotemporal Neural Network on Graph (Graph-STNN). Specifically, the proposed Graph-STNN extracts the spatial feature using graph convolutional network (GCN) and the temporal feature using temporal convolutional network (TCN). Then, gated recurrent unit (GRU) is performed to synthesize the spatial feature, the temporal feature, and the historical state information into the current state feature. Combined with DRL, our Graph-STNN helps estimation of the optimal target point through extracted hybrid features. The simulation experiment shows that our approach is more effective than the GCN-based approach and the information entropy-based approach. Moreover, Graph-STNN also performs better generalization ability than GCN-based, information entropy-based, and random methods. Finally, we validate our approach on the simulation platform Stage with the actual robot model. |
| first_indexed | 2024-03-10T07:56:02Z |
| format | Article |
| id | doaj.art-6064d63c4aa04ec28c499fc9b295f8f8 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T07:56:02Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6064d63c4aa04ec28c499fc9b295f8f82023-11-22T11:50:30ZengMDPI AGApplied Sciences2076-34172021-09-011118829910.3390/app11188299Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on GraphZhiwen Zhang0Chenghao Shi1Pengming Zhu2Zhiwen Zeng3Hui Zhang4Robotics Research Center, College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaRobotics Research Center, College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaRobotics Research Center, College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaRobotics Research Center, College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaRobotics Research Center, College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, ChinaIn this paper, we address the problem of autonomous exploration in unknown environments for ground mobile robots with deep reinforcement learning (DRL). To effectively explore unknown environments, we construct an exploration graph considering historical trajectories, frontier waypoints, landmarks, and obstacles. Meanwhile, to take full advantage of the spatiotemporal feature and historical information in the autonomous exploration task, we propose a novel network called Spatiotemporal Neural Network on Graph (Graph-STNN). Specifically, the proposed Graph-STNN extracts the spatial feature using graph convolutional network (GCN) and the temporal feature using temporal convolutional network (TCN). Then, gated recurrent unit (GRU) is performed to synthesize the spatial feature, the temporal feature, and the historical state information into the current state feature. Combined with DRL, our Graph-STNN helps estimation of the optimal target point through extracted hybrid features. The simulation experiment shows that our approach is more effective than the GCN-based approach and the information entropy-based approach. Moreover, Graph-STNN also performs better generalization ability than GCN-based, information entropy-based, and random methods. Finally, we validate our approach on the simulation platform Stage with the actual robot model.https://www.mdpi.com/2076-3417/11/18/8299autonomous explorationdeep reinforcement learningspatiotemporal informationgraph convolutional networktemporal convolutional networkgated recurrent unit |
| spellingShingle | Zhiwen Zhang Chenghao Shi Pengming Zhu Zhiwen Zeng Hui Zhang Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph Applied Sciences autonomous exploration deep reinforcement learning spatiotemporal information graph convolutional network temporal convolutional network gated recurrent unit |
| title | Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph |
| title_full | Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph |
| title_fullStr | Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph |
| title_full_unstemmed | Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph |
| title_short | Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph |
| title_sort | autonomous exploration of mobile robots via deep reinforcement learning based on spatiotemporal information on graph |
| topic | autonomous exploration deep reinforcement learning spatiotemporal information graph convolutional network temporal convolutional network gated recurrent unit |
| url | https://www.mdpi.com/2076-3417/11/18/8299 |
| work_keys_str_mv | AT zhiwenzhang autonomousexplorationofmobilerobotsviadeepreinforcementlearningbasedonspatiotemporalinformationongraph AT chenghaoshi autonomousexplorationofmobilerobotsviadeepreinforcementlearningbasedonspatiotemporalinformationongraph AT pengmingzhu autonomousexplorationofmobilerobotsviadeepreinforcementlearningbasedonspatiotemporalinformationongraph AT zhiwenzeng autonomousexplorationofmobilerobotsviadeepreinforcementlearningbasedonspatiotemporalinformationongraph AT huizhang autonomousexplorationofmobilerobotsviadeepreinforcementlearningbasedonspatiotemporalinformationongraph |