Advancing In-hand Dexterous Manipulation via Machine Learning
Robots are becoming better at navigating and moving around, but they still struggle with using tools, which severely limits their usefulness for household tasks. Using tools requires dexterously manipulating everyday objects like hammers, scissors, knives, screwdrivers, etc. While simple for humans,...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156350 |
| _version_ | 1826196746743578624 |
|---|---|
| author | Chen, Tao |
| author2 | Agrawal, Pulkit |
| author_facet | Agrawal, Pulkit Chen, Tao |
| author_sort | Chen, Tao |
| collection | MIT |
| description | Robots are becoming better at navigating and moving around, but they still struggle with using tools, which severely limits their usefulness for household tasks. Using tools requires dexterously manipulating everyday objects like hammers, scissors, knives, screwdrivers, etc. While simple for humans, manipulating everyday objects remains a long-standing challenge that requires breakthroughs in robotic hardware, sensing, perception, and control algorithms. This thesis proposes machine learning techniques that substantially improve the state-ofthe-art performance of dexterous manipulation controllers. It focuses specifically on in-hand object reorientation tasks. Previous works on this problem had limitations like using expensive sensors or hands, only working for a few objects, requiring the hand to face upward, slow object motion, etc. This thesis goes a step further by enabling a low-cost robot hand to dynamically reorient diverse objects in mid-air with the hand facing downward using an inexpensive depth camera. To train such a system, the thesis proposes techniques for robots to learn to reorient objects with a downward-facing hand in the air. It also proposes multiple techniques to improve the time efficiency of the learning algorithms. Additionally, it discusses how to reduce the gap between simulation and reality so that controllers trained in simulation can transfer directly to real systems. Furthermore, the thesis explores the use of tactile sensors in dexterous manipulation. It concludes with a discussion of the current system’s issues and outlines future research directions for dexterous manipulation. |
| first_indexed | 2024-09-23T10:37:24Z |
| format | Thesis |
| id | mit-1721.1/156350 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:37:24Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1563502024-08-22T03:46:21Z Advancing In-hand Dexterous Manipulation via Machine Learning Chen, Tao Agrawal, Pulkit Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Robots are becoming better at navigating and moving around, but they still struggle with using tools, which severely limits their usefulness for household tasks. Using tools requires dexterously manipulating everyday objects like hammers, scissors, knives, screwdrivers, etc. While simple for humans, manipulating everyday objects remains a long-standing challenge that requires breakthroughs in robotic hardware, sensing, perception, and control algorithms. This thesis proposes machine learning techniques that substantially improve the state-ofthe-art performance of dexterous manipulation controllers. It focuses specifically on in-hand object reorientation tasks. Previous works on this problem had limitations like using expensive sensors or hands, only working for a few objects, requiring the hand to face upward, slow object motion, etc. This thesis goes a step further by enabling a low-cost robot hand to dynamically reorient diverse objects in mid-air with the hand facing downward using an inexpensive depth camera. To train such a system, the thesis proposes techniques for robots to learn to reorient objects with a downward-facing hand in the air. It also proposes multiple techniques to improve the time efficiency of the learning algorithms. Additionally, it discusses how to reduce the gap between simulation and reality so that controllers trained in simulation can transfer directly to real systems. Furthermore, the thesis explores the use of tactile sensors in dexterous manipulation. It concludes with a discussion of the current system’s issues and outlines future research directions for dexterous manipulation. Ph.D. 2024-08-21T18:58:48Z 2024-08-21T18:58:48Z 2024-05 2024-07-10T13:01:27.939Z Thesis https://hdl.handle.net/1721.1/156350 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Chen, Tao Advancing In-hand Dexterous Manipulation via Machine Learning |
| title | Advancing In-hand Dexterous Manipulation via Machine Learning |
| title_full | Advancing In-hand Dexterous Manipulation via Machine Learning |
| title_fullStr | Advancing In-hand Dexterous Manipulation via Machine Learning |
| title_full_unstemmed | Advancing In-hand Dexterous Manipulation via Machine Learning |
| title_short | Advancing In-hand Dexterous Manipulation via Machine Learning |
| title_sort | advancing in hand dexterous manipulation via machine learning |
| url | https://hdl.handle.net/1721.1/156350 |
| work_keys_str_mv | AT chentao advancinginhanddexterousmanipulationviamachinelearning |