Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation
What is the “state” of a pile of objects? Collections of standard Lagrangian states ideally describe a pile, but it is impractical to estimate Lagrangian states directly from images to use for visuomotor feedback control. Given this burden of state estimation, is there a practical alternative repres...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/143378 |
| _version_ | 1826212834401320960 |
|---|---|
| author | Suh, Hyung Ju Terry |
| author2 | Tedrake, Russell L. |
| author_facet | Tedrake, Russell L. Suh, Hyung Ju Terry |
| author_sort | Suh, Hyung Ju Terry |
| collection | MIT |
| description | What is the “state” of a pile of objects? Collections of standard Lagrangian states ideally describe a pile, but it is impractical to estimate Lagrangian states directly from images to use for visuomotor feedback control. Given this burden of state estimation, is there a practical alternative representation that lies closer to our observations? In addition, how can we build predictive models over such representations that can be useful for their task-free generality? In the first chapter of this thesis, we investigate using the image observation directly as state, and compare different models that can be useful over this space of representations. We surprisingly find that completely linear models that describe the evolution of images outperform naive deep models, and perform in par with models that work over particle-space representations. In the next chapter, we analyze and describe the reason for this inductive bias of linear models by describing the pixel space as a space of measures, and show limitations of this approach outside of object pile manipulation. In the final chapter of this thesis, we present a more general solution to image-based control based on doing model-based Reinforcement Learning on the sufficient statistics of a task, which we call Approximate Information States (AIS). We demonstrate that when the model does not have sufficient inductive bias, model-based reinforcement learning is prone to two important pitfalls: distribution shift, and optimization exploiting model error. These problems are tackled through online learning, and risk-aware control that penalizes the variance of the model ensemble. |
| first_indexed | 2024-09-23T15:38:58Z |
| format | Thesis |
| id | mit-1721.1/143378 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T15:38:58Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1433782022-06-16T03:08:08Z Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation Suh, Hyung Ju Terry Tedrake, Russell L. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science What is the “state” of a pile of objects? Collections of standard Lagrangian states ideally describe a pile, but it is impractical to estimate Lagrangian states directly from images to use for visuomotor feedback control. Given this burden of state estimation, is there a practical alternative representation that lies closer to our observations? In addition, how can we build predictive models over such representations that can be useful for their task-free generality? In the first chapter of this thesis, we investigate using the image observation directly as state, and compare different models that can be useful over this space of representations. We surprisingly find that completely linear models that describe the evolution of images outperform naive deep models, and perform in par with models that work over particle-space representations. In the next chapter, we analyze and describe the reason for this inductive bias of linear models by describing the pixel space as a space of measures, and show limitations of this approach outside of object pile manipulation. In the final chapter of this thesis, we present a more general solution to image-based control based on doing model-based Reinforcement Learning on the sufficient statistics of a task, which we call Approximate Information States (AIS). We demonstrate that when the model does not have sufficient inductive bias, model-based reinforcement learning is prone to two important pitfalls: distribution shift, and optimization exploiting model error. These problems are tackled through online learning, and risk-aware control that penalizes the variance of the model ensemble. S.M. 2022-06-15T13:16:30Z 2022-06-15T13:16:30Z 2022-02 2022-03-04T20:59:52.218Z Thesis https://hdl.handle.net/1721.1/143378 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Suh, Hyung Ju Terry Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title | Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title_full | Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title_fullStr | Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title_full_unstemmed | Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title_short | Predictive Models for Visuomotor Feedback Control in Object Pile Manipulation |
| title_sort | predictive models for visuomotor feedback control in object pile manipulation |
| url | https://hdl.handle.net/1721.1/143378 |
| work_keys_str_mv | AT suhhyungjuterry predictivemodelsforvisuomotorfeedbackcontrolinobjectpilemanipulation |