Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge
Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC) [1]. A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occl...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2019
|
| Online Access: | http://hdl.handle.net/1721.1/121121 https://orcid.org/0000-0002-8954-2310 https://orcid.org/0000-0002-1119-4512 |
| _version_ | 1811084034409758720 |
|---|---|
| author | Zeng, Andy Song, Shuran Suo, Daniel Walker, Ed Xiao, Jianxiong Yu, Kuan-Ting Rodriguez Garcia, Alberto |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Zeng, Andy Song, Shuran Suo, Daniel Walker, Ed Xiao, Jianxiong Yu, Kuan-Ting Rodriguez Garcia, Alberto |
| author_sort | Zeng, Andy |
| collection | MIT |
| description | Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC) [1]. A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multiview RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th-place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu/ |
| first_indexed | 2024-09-23T12:43:33Z |
| format | Article |
| id | mit-1721.1/121121 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T12:43:33Z |
| publishDate | 2019 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1211212022-10-01T10:46:49Z Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge Zeng, Andy Song, Shuran Suo, Daniel Walker, Ed Xiao, Jianxiong Yu, Kuan-Ting Rodriguez Garcia, Alberto Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Mechanical Engineering Yu, Kuan-Ting Rodriguez Garcia, Alberto Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC) [1]. A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multiview RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th-place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu/ 2019-03-29T19:46:15Z 2019-03-29T19:46:15Z 2017-07 2017-06 2018-12-17T18:13:34Z Article http://purl.org/eprint/type/ConferencePaper 978-1-5090-4633-1 http://hdl.handle.net/1721.1/121121 Zeng, Andy, Kuan-Ting Yu, Shuran Song, Daniel Suo, Ed Walker, Alberto Rodriguez, and Jianxiong Xiao. “Multi-View Self-Supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge.” 2017 IEEE International Conference on Robotics and Automation (ICRA), 29 May - 3 July, 2017, Singapore, Singapore, IEEE, 2017. © 2017 IEEE https://orcid.org/0000-0002-8954-2310 https://orcid.org/0000-0002-1119-4512 http://dx.doi.org/10.1109/ICRA.2017.7989165 2017 IEEE International Conference on Robotics and Automation (ICRA) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Zeng, Andy Song, Shuran Suo, Daniel Walker, Ed Xiao, Jianxiong Yu, Kuan-Ting Rodriguez Garcia, Alberto Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title | Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title_full | Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title_fullStr | Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title_full_unstemmed | Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title_short | Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge |
| title_sort | multi view self supervised deep learning for 6d pose estimation in the amazon picking challenge |
| url | http://hdl.handle.net/1721.1/121121 https://orcid.org/0000-0002-8954-2310 https://orcid.org/0000-0002-1119-4512 |
| work_keys_str_mv | AT zengandy multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT songshuran multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT suodaniel multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT walkered multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT xiaojianxiong multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT yukuanting multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge AT rodriguezgarciaalberto multiviewselfsuperviseddeeplearningfor6dposeestimationintheamazonpickingchallenge |