SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
Existing scene understanding datasets contain only a limited set of views of a place, and they lack representations of complete 3D spaces. In this paper, we introduce SUN3D, a large-scale RGB-D video database with camera pose and object labels, capturing the full 3D extent of many places. The tasks...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2014
|
| Online Access: | http://hdl.handle.net/1721.1/91003 https://orcid.org/0000-0001-9020-9593 https://orcid.org/0000-0003-4915-0256 |
| _version_ | 1811073994716086272 |
|---|---|
| author | Xiao, Jianxiong Torralba, Antonio Owens, Andrew Hale |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Xiao, Jianxiong Torralba, Antonio Owens, Andrew Hale |
| author_sort | Xiao, Jianxiong |
| collection | MIT |
| description | Existing scene understanding datasets contain only a limited set of views of a place, and they lack representations of complete 3D spaces. In this paper, we introduce SUN3D, a large-scale RGB-D video database with camera pose and object labels, capturing the full 3D extent of many places. The tasks that go into constructing such a dataset are difficult in isolation -- hand-labeling videos is painstaking, and structure from motion (SfM) is unreliable for large spaces. But if we combine them together, we make the dataset construction task much easier. First, we introduce an intuitive labeling tool that uses a partial reconstruction to propagate labels from one frame to another. Then we use the object labels to fix errors in the reconstruction. For this, we introduce a generalization of bundle adjustment that incorporates object-to-object correspondences. This algorithm works by constraining points for the same object from different frames to lie inside a fixed-size bounding box, parameterized by its rotation and translation. The SUN3D database, the source code for the generalized bundle adjustment, and the web-based 3D annotation tool are all available at http://sun3d.cs.princeton.edu. |
| first_indexed | 2024-09-23T09:41:24Z |
| format | Article |
| id | mit-1721.1/91003 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T09:41:24Z |
| publishDate | 2014 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/910032022-09-26T13:05:30Z SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels Xiao, Jianxiong Torralba, Antonio Owens, Andrew Hale Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Owens, Andrew Hale Torralba, Antonio Existing scene understanding datasets contain only a limited set of views of a place, and they lack representations of complete 3D spaces. In this paper, we introduce SUN3D, a large-scale RGB-D video database with camera pose and object labels, capturing the full 3D extent of many places. The tasks that go into constructing such a dataset are difficult in isolation -- hand-labeling videos is painstaking, and structure from motion (SfM) is unreliable for large spaces. But if we combine them together, we make the dataset construction task much easier. First, we introduce an intuitive labeling tool that uses a partial reconstruction to propagate labels from one frame to another. Then we use the object labels to fix errors in the reconstruction. For this, we introduce a generalization of bundle adjustment that incorporates object-to-object correspondences. This algorithm works by constraining points for the same object from different frames to lie inside a fixed-size bounding box, parameterized by its rotation and translation. The SUN3D database, the source code for the generalized bundle adjustment, and the web-based 3D annotation tool are all available at http://sun3d.cs.princeton.edu. American Society for Engineering Education. National Defense Science and Engineering Graduate Fellowship United States. Office of Naval Research. Multidisciplinary University Research Initiative (N000141010933) 2014-10-20T18:26:14Z 2014-10-20T18:26:14Z 2013-12 Article http://purl.org/eprint/type/ConferencePaper 978-1-4799-2840-8 1550-5499 http://hdl.handle.net/1721.1/91003 Xiao, Jianxiong, Andrew Owens, and Antonio Torralba. “SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels.” 2013 IEEE International Conference on Computer Vision (December 2013). https://orcid.org/0000-0001-9020-9593 https://orcid.org/0000-0003-4915-0256 en_US http://dx.doi.org/10.1109/ICCV.2013.458 Proceedings of the 2013 IEEE International Conference on Computer Vision Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Xiao, Jianxiong Torralba, Antonio Owens, Andrew Hale SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title | SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title_full | SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title_fullStr | SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title_full_unstemmed | SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title_short | SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels |
| title_sort | sun3d a database of big spaces reconstructed using sfm and object labels |
| url | http://hdl.handle.net/1721.1/91003 https://orcid.org/0000-0001-9020-9593 https://orcid.org/0000-0003-4915-0256 |
| work_keys_str_mv | AT xiaojianxiong sun3dadatabaseofbigspacesreconstructedusingsfmandobjectlabels AT torralbaantonio sun3dadatabaseofbigspacesreconstructedusingsfmandobjectlabels AT owensandrewhale sun3dadatabaseofbigspacesreconstructedusingsfmandobjectlabels |