Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space
In this paper, we propose a goal-oriented obstacle avoidance navigation system based on deep reinforcement learning that uses depth information in scenes, as well as goal position in polar coordinates as state inputs. The control signals for robot motion are output in a continuous action space. We d...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-02-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/9/3/411 |
| _version_ | 1798025824630210560 |
|---|---|
| author | Reinis Cimurs Jin Han Lee Il Hong Suh |
| author_facet | Reinis Cimurs Jin Han Lee Il Hong Suh |
| author_sort | Reinis Cimurs |
| collection | DOAJ |
| description | In this paper, we propose a goal-oriented obstacle avoidance navigation system based on deep reinforcement learning that uses depth information in scenes, as well as goal position in polar coordinates as state inputs. The control signals for robot motion are output in a continuous action space. We devise a deep deterministic policy gradient network with the inclusion of depth-wise separable convolution layers to process the large amounts of sequential depth image information. The goal-oriented obstacle avoidance navigation is performed without prior knowledge of the environment or a map. We show that through the proposed deep reinforcement learning network, a goal-oriented collision avoidance model can be trained end-to-end without manual tuning or supervision by a human operator. We train our model in a simulation, and the resulting network is directly transferred to other environments. Experiments show the capability of the trained network to navigate safely around obstacles and arrive at the designated goal positions in the simulation, as well as in the real world. The proposed method exhibits higher reliability than the compared approaches when navigating around obstacles with complex shapes. The experiments show that the approach is capable of avoiding not only static, but also dynamic obstacles. |
| first_indexed | 2024-04-11T18:25:08Z |
| format | Article |
| id | doaj.art-34485e2ae6dc422694c7cfdf6334aaac |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-04-11T18:25:08Z |
| publishDate | 2020-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-34485e2ae6dc422694c7cfdf6334aaac2022-12-22T04:09:39ZengMDPI AGElectronics2079-92922020-02-019341110.3390/electronics9030411electronics9030411Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action SpaceReinis Cimurs0Jin Han Lee1Il Hong Suh2Department of Intelligent Robot Engineering, Hanyang University, Seoul 04763, KoreaDepartment of Electronics and Computer Engineering, Hanyang University, Seoul 04763, KoreaDepartment of Electronics and Computer Engineering, Hanyang University, Seoul 04763, KoreaIn this paper, we propose a goal-oriented obstacle avoidance navigation system based on deep reinforcement learning that uses depth information in scenes, as well as goal position in polar coordinates as state inputs. The control signals for robot motion are output in a continuous action space. We devise a deep deterministic policy gradient network with the inclusion of depth-wise separable convolution layers to process the large amounts of sequential depth image information. The goal-oriented obstacle avoidance navigation is performed without prior knowledge of the environment or a map. We show that through the proposed deep reinforcement learning network, a goal-oriented collision avoidance model can be trained end-to-end without manual tuning or supervision by a human operator. We train our model in a simulation, and the resulting network is directly transferred to other environments. Experiments show the capability of the trained network to navigate safely around obstacles and arrive at the designated goal positions in the simulation, as well as in the real world. The proposed method exhibits higher reliability than the compared approaches when navigating around obstacles with complex shapes. The experiments show that the approach is capable of avoiding not only static, but also dynamic obstacles.https://www.mdpi.com/2079-9292/9/3/411deep reinforcement learningobstacle avoidancemap-less vector navigationmixed-input network |
| spellingShingle | Reinis Cimurs Jin Han Lee Il Hong Suh Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space Electronics deep reinforcement learning obstacle avoidance map-less vector navigation mixed-input network |
| title | Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space |
| title_full | Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space |
| title_fullStr | Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space |
| title_full_unstemmed | Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space |
| title_short | Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space |
| title_sort | goal oriented obstacle avoidance with deep reinforcement learning in continuous action space |
| topic | deep reinforcement learning obstacle avoidance map-less vector navigation mixed-input network |
| url | https://www.mdpi.com/2079-9292/9/3/411 |
| work_keys_str_mv | AT reiniscimurs goalorientedobstacleavoidancewithdeepreinforcementlearningincontinuousactionspace AT jinhanlee goalorientedobstacleavoidancewithdeepreinforcementlearningincontinuousactionspace AT ilhongsuh goalorientedobstacleavoidancewithdeepreinforcementlearningincontinuousactionspace |