Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more prec...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/130010 |
| _version_ | 1811079232160268288 |
|---|---|
| author | Zeng, Andy Song, Shuran Welker, Stefan Lee, Johnny Rodriguez Garcia, Alberto Funkhouser, Thomas |
| author2 | Massachusetts Institute of Technology. Department of Mechanical Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Mechanical Engineering Zeng, Andy Song, Shuran Welker, Stefan Lee, Johnny Rodriguez Garcia, Alberto Funkhouser, Thomas |
| author_sort | Zeng, Andy |
| collection | MIT |
| description | Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end-effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors even amid challenging cases of tightly packed clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. Qualitative results (videos), code, pre-trained models, and simulation environments are available at http://vpg.cs.princeton.edu . |
| first_indexed | 2024-09-23T11:11:56Z |
| format | Article |
| id | mit-1721.1/130010 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:11:56Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1300102022-10-01T01:58:31Z Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning Zeng, Andy Song, Shuran Welker, Stefan Lee, Johnny Rodriguez Garcia, Alberto Funkhouser, Thomas Massachusetts Institute of Technology. Department of Mechanical Engineering Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end-effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors even amid challenging cases of tightly packed clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. Qualitative results (videos), code, pre-trained models, and simulation environments are available at http://vpg.cs.princeton.edu . NSF (Grant VEC-1539014/1539099) 2021-02-26T16:49:08Z 2021-02-26T16:49:08Z 2019-01 2018-10 2020-07-31T18:11:55Z Article http://purl.org/eprint/type/ConferencePaper 9781538680940 2153-0866 https://hdl.handle.net/1721.1/130010 Zeng, Andy et al. "Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning." IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2018, Madrid, Spain, Institute of Electrical and Electronics Engineers, January 2019. © 2018 IEEE en http://dx.doi.org/10.1109/iros.2018.8593986 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Zeng, Andy Song, Shuran Welker, Stefan Lee, Johnny Rodriguez Garcia, Alberto Funkhouser, Thomas Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title | Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title_full | Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title_fullStr | Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title_full_unstemmed | Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title_short | Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning |
| title_sort | learning synergies between pushing and grasping with self supervised deep reinforcement learning |
| url | https://hdl.handle.net/1721.1/130010 |
| work_keys_str_mv | AT zengandy learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning AT songshuran learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning AT welkerstefan learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning AT leejohnny learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning AT rodriguezgarciaalberto learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning AT funkhouserthomas learningsynergiesbetweenpushingandgraspingwithselfsuperviseddeepreinforcementlearning |