Stage-Wise Learning of Reaching Using Little Prior Knowledge
In some manipulation robotics environments, because of the difficulty of precisely modeling dynamics and computing features which describe well the variety of scene appearances, hand-programming a robot behavior is often intractable. Deep reinforcement learning methods partially alleviate this probl...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-10-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/frobt.2018.00110/full |
| _version_ | 1818495784511864832 |
|---|---|
| author | François de La Bourdonnaye Céline Teulière Jochen Triesch Thierry Chateau |
| author_facet | François de La Bourdonnaye Céline Teulière Jochen Triesch Thierry Chateau |
| author_sort | François de La Bourdonnaye |
| collection | DOAJ |
| description | In some manipulation robotics environments, because of the difficulty of precisely modeling dynamics and computing features which describe well the variety of scene appearances, hand-programming a robot behavior is often intractable. Deep reinforcement learning methods partially alleviate this problem in that they can dispense with hand-crafted features for the state representation and do not need pre-computed dynamics. However, they often use prior information in the task definition in the form of shaping rewards which guide the robot toward goal state areas but require engineering or human supervision and can lead to sub-optimal behavior. In this work we consider a complex robot reaching task with a large range of initial object positions and initial arm positions and propose a new learning approach with minimal supervision. Inspired by developmental robotics, our method consists of a weakly-supervised stage-wise procedure of three tasks. First, the robot learns to fixate the object with a 2-camera system. Second, it learns hand-eye coordination by learning to fixate its end-effector. Third, using the knowledge acquired in the previous steps, it learns to reach the object at different positions and from a large set of initial robot joint angles. Experiments in a simulated environment show that our stage-wise framework yields similar reaching performances, compared with a supervised setting without using kinematic models, hand-crafted features, calibration parameters or supervised visual modules. |
| first_indexed | 2024-12-10T18:25:15Z |
| format | Article |
| id | doaj.art-fcb0fccbfc5d4d90b11128404fa5d5e5 |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-12-10T18:25:15Z |
| publishDate | 2018-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-fcb0fccbfc5d4d90b11128404fa5d5e52022-12-22T01:38:06ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442018-10-01510.3389/frobt.2018.00110410106Stage-Wise Learning of Reaching Using Little Prior KnowledgeFrançois de La Bourdonnaye0Céline Teulière1Jochen Triesch2Thierry Chateau3CNRS, SIGMA Clermont, Institut Pascal, Université Clermont Auvergne, Clermont-Ferrand, FranceCNRS, SIGMA Clermont, Institut Pascal, Université Clermont Auvergne, Clermont-Ferrand, FranceFrankfurt Institute for Advanced Studies, Frankfurt am Main, GermanyCNRS, SIGMA Clermont, Institut Pascal, Université Clermont Auvergne, Clermont-Ferrand, FranceIn some manipulation robotics environments, because of the difficulty of precisely modeling dynamics and computing features which describe well the variety of scene appearances, hand-programming a robot behavior is often intractable. Deep reinforcement learning methods partially alleviate this problem in that they can dispense with hand-crafted features for the state representation and do not need pre-computed dynamics. However, they often use prior information in the task definition in the form of shaping rewards which guide the robot toward goal state areas but require engineering or human supervision and can lead to sub-optimal behavior. In this work we consider a complex robot reaching task with a large range of initial object positions and initial arm positions and propose a new learning approach with minimal supervision. Inspired by developmental robotics, our method consists of a weakly-supervised stage-wise procedure of three tasks. First, the robot learns to fixate the object with a 2-camera system. Second, it learns hand-eye coordination by learning to fixate its end-effector. Third, using the knowledge acquired in the previous steps, it learns to reach the object at different positions and from a large set of initial robot joint angles. Experiments in a simulated environment show that our stage-wise framework yields similar reaching performances, compared with a supervised setting without using kinematic models, hand-crafted features, calibration parameters or supervised visual modules.https://www.frontiersin.org/article/10.3389/frobt.2018.00110/fulldeep reinforcement learningweakly-supervisedstage-wise learningmanipulation roboticshierarchical learning |
| spellingShingle | François de La Bourdonnaye Céline Teulière Jochen Triesch Thierry Chateau Stage-Wise Learning of Reaching Using Little Prior Knowledge Frontiers in Robotics and AI deep reinforcement learning weakly-supervised stage-wise learning manipulation robotics hierarchical learning |
| title | Stage-Wise Learning of Reaching Using Little Prior Knowledge |
| title_full | Stage-Wise Learning of Reaching Using Little Prior Knowledge |
| title_fullStr | Stage-Wise Learning of Reaching Using Little Prior Knowledge |
| title_full_unstemmed | Stage-Wise Learning of Reaching Using Little Prior Knowledge |
| title_short | Stage-Wise Learning of Reaching Using Little Prior Knowledge |
| title_sort | stage wise learning of reaching using little prior knowledge |
| topic | deep reinforcement learning weakly-supervised stage-wise learning manipulation robotics hierarchical learning |
| url | https://www.frontiersin.org/article/10.3389/frobt.2018.00110/full |
| work_keys_str_mv | AT francoisdelabourdonnaye stagewiselearningofreachingusinglittlepriorknowledge AT celineteuliere stagewiselearningofreachingusinglittlepriorknowledge AT jochentriesch stagewiselearningofreachingusinglittlepriorknowledge AT thierrychateau stagewiselearningofreachingusinglittlepriorknowledge |