Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
© 2018 IEEE. We study 3D shape modeling from a single image and make contributions to it in three aspects. First, we present Pix3D, a large-scale benchmark of diverse image-shape pairs with pixel-level 2D-3D alignment. Pix3D has wide applications in shape-related tasks including reconstruction, retr...
| Main Authors: | , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
2022
|
| Online Access: | https://hdl.handle.net/1721.1/132170.2 |
| _version_ | 1811082015453216768 |
|---|---|
| author | Sun, Xingyuan Wu, Jiajun Zhang, Xiuming Zhang, Zhoutong Zhang, Chengkai Xue, Tianfan Tenenbaum, Joshua B Freeman, William T |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Sun, Xingyuan Wu, Jiajun Zhang, Xiuming Zhang, Zhoutong Zhang, Chengkai Xue, Tianfan Tenenbaum, Joshua B Freeman, William T |
| author_sort | Sun, Xingyuan |
| collection | MIT |
| description | © 2018 IEEE. We study 3D shape modeling from a single image and make contributions to it in three aspects. First, we present Pix3D, a large-scale benchmark of diverse image-shape pairs with pixel-level 2D-3D alignment. Pix3D has wide applications in shape-related tasks including reconstruction, retrieval, viewpoint estimation, etc. Building such a large-scale dataset, however, is highly challenging; existing datasets either contain only synthetic data, or lack precise alignment between 2D images and 3D shapes, or only have a small number of images. Second, we calibrate the evaluation criteria for 3D shape reconstruction through behavioral studies, and use them to objectively and systematically benchmark cutting-edge reconstruction algorithms on Pix3D. Third, we design a novel model that simultaneously performs 3D reconstruction and pose estimation; our multi-task learning approach achieves state-of-the-art performance on both tasks. |
| first_indexed | 2024-09-23T11:56:02Z |
| format | Article |
| id | mit-1721.1/132170.2 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:56:02Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | mit-1721.1/132170.22022-08-04T18:56:32Z Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling Sun, Xingyuan Wu, Jiajun Zhang, Xiuming Zhang, Zhoutong Zhang, Chengkai Xue, Tianfan Tenenbaum, Joshua B Freeman, William T Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences © 2018 IEEE. We study 3D shape modeling from a single image and make contributions to it in three aspects. First, we present Pix3D, a large-scale benchmark of diverse image-shape pairs with pixel-level 2D-3D alignment. Pix3D has wide applications in shape-related tasks including reconstruction, retrieval, viewpoint estimation, etc. Building such a large-scale dataset, however, is highly challenging; existing datasets either contain only synthetic data, or lack precise alignment between 2D images and 3D shapes, or only have a small number of images. Second, we calibrate the evaluation criteria for 3D shape reconstruction through behavioral studies, and use them to objectively and systematically benchmark cutting-edge reconstruction algorithms on Pix3D. Third, we design a novel model that simultaneously performs 3D reconstruction and pose estimation; our multi-task learning approach achieves state-of-the-art performance on both tasks. 2022-08-04T18:56:31Z 2021-09-20T18:21:14Z 2022-08-04T18:56:31Z 2018 2019-05-23T15:58:03Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/132170.2 en 10.1109/CVPR.2018.00314 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/octet-stream MIT web domain |
| spellingShingle | Sun, Xingyuan Wu, Jiajun Zhang, Xiuming Zhang, Zhoutong Zhang, Chengkai Xue, Tianfan Tenenbaum, Joshua B Freeman, William T Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title | Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title_full | Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title_fullStr | Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title_full_unstemmed | Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title_short | Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling |
| title_sort | pix3d dataset and methods for single image 3d shape modeling |
| url | https://hdl.handle.net/1721.1/132170.2 |
| work_keys_str_mv | AT sunxingyuan pix3ddatasetandmethodsforsingleimage3dshapemodeling AT wujiajun pix3ddatasetandmethodsforsingleimage3dshapemodeling AT zhangxiuming pix3ddatasetandmethodsforsingleimage3dshapemodeling AT zhangzhoutong pix3ddatasetandmethodsforsingleimage3dshapemodeling AT zhangchengkai pix3ddatasetandmethodsforsingleimage3dshapemodeling AT xuetianfan pix3ddatasetandmethodsforsingleimage3dshapemodeling AT tenenbaumjoshuab pix3ddatasetandmethodsforsingleimage3dshapemodeling AT freemanwilliamt pix3ddatasetandmethodsforsingleimage3dshapemodeling |