DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions
We study the problem of learning physical object representations for robot manipulation. Understanding object physics is critical for successful object manipulation, but also challenging because physical object properties can rarely be inferred from the object’s static appearance. In this paper,...
| Main Authors: | , , , , |
|---|---|
| מחברים אחרים: | |
| פורמט: | Article |
| שפה: | English |
| יצא לאור: |
Robotics: Science and Systems Foundation
2021
|
| גישה מקוונת: | https://hdl.handle.net/1721.1/138341 |
| _version_ | 1826188144843685888 |
|---|---|
| author | Xu, Zhenjia Wu, Jiajun Zeng, Andy Tenenbaum, Joshua Song, Shuran |
| author2 | Center for Brains, Minds, and Machines |
| author_facet | Center for Brains, Minds, and Machines Xu, Zhenjia Wu, Jiajun Zeng, Andy Tenenbaum, Joshua Song, Shuran |
| author_sort | Xu, Zhenjia |
| collection | MIT |
| description | We study the problem of learning physical object
representations for robot manipulation. Understanding object
physics is critical for successful object manipulation, but also
challenging because physical object properties can rarely be
inferred from the object’s static appearance. In this paper, we
propose DensePhysNet, a system that actively executes a sequence
of dynamic interactions (e.g., sliding and colliding), and uses a
deep predictive model over its visual observations to learn dense,
pixel-wise representations that reflect the physical properties of
observed objects. Our experiments in both simulation and real
settings demonstrate that the learned representations carry rich
physical information, and can directly be used to decode physical
object properties such as friction and mass. The use of dense
representation enables DensePhysNet to generalize well to novel
scenes with more objects than in training. With knowledge of
object physics, the learned representation also leads to more
accurate and efficient manipulation in downstream tasks than
the state-of-the-art. |
| first_indexed | 2024-09-23T07:55:16Z |
| format | Article |
| id | mit-1721.1/138341 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T07:55:16Z |
| publishDate | 2021 |
| publisher | Robotics: Science and Systems Foundation |
| record_format | dspace |
| spelling | mit-1721.1/1383412023-01-30T21:26:54Z DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions Xu, Zhenjia Wu, Jiajun Zeng, Andy Tenenbaum, Joshua Song, Shuran Center for Brains, Minds, and Machines We study the problem of learning physical object representations for robot manipulation. Understanding object physics is critical for successful object manipulation, but also challenging because physical object properties can rarely be inferred from the object’s static appearance. In this paper, we propose DensePhysNet, a system that actively executes a sequence of dynamic interactions (e.g., sliding and colliding), and uses a deep predictive model over its visual observations to learn dense, pixel-wise representations that reflect the physical properties of observed objects. Our experiments in both simulation and real settings demonstrate that the learned representations carry rich physical information, and can directly be used to decode physical object properties such as friction and mass. The use of dense representation enables DensePhysNet to generalize well to novel scenes with more objects than in training. With knowledge of object physics, the learned representation also leads to more accurate and efficient manipulation in downstream tasks than the state-of-the-art. 2021-12-07T13:49:46Z 2021-12-07T13:49:46Z 2019 2021-12-07T13:46:32Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/138341 Xu, Zhenjia, Wu, Jiajun, Zeng, Andy, Tenenbaum, Joshua and Song, Shuran. 2019. "DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions." Robotics: Science and Systems XV. en 10.15607/RSS.2019.XV.046 Robotics: Science and Systems XV Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Robotics: Science and Systems Foundation arXiv |
| spellingShingle | Xu, Zhenjia Wu, Jiajun Zeng, Andy Tenenbaum, Joshua Song, Shuran DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title | DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title_full | DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title_fullStr | DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title_full_unstemmed | DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title_short | DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions |
| title_sort | densephysnet learning dense physical object representations via multi step dynamic interactions |
| url | https://hdl.handle.net/1721.1/138341 |
| work_keys_str_mv | AT xuzhenjia densephysnetlearningdensephysicalobjectrepresentationsviamultistepdynamicinteractions AT wujiajun densephysnetlearningdensephysicalobjectrepresentationsviamultistepdynamicinteractions AT zengandy densephysnetlearningdensephysicalobjectrepresentationsviamultistepdynamicinteractions AT tenenbaumjoshua densephysnetlearningdensephysicalobjectrepresentationsviamultistepdynamicinteractions AT songshuran densephysnetlearningdensephysicalobjectrepresentationsviamultistepdynamicinteractions |