NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects
Robotics: Science and Systems Jul 10 – Jul 14, 2023 Daegu, Republic of Korea
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Robotics: Science and Systems Foundation
2024
|
| Online Access: | https://hdl.handle.net/1721.1/153745 |
| _version_ | 1811089083194146816 |
|---|---|
| author | Fu, Jiahui Du, Yilun Singh, Kurran Tenenbaum, Joshua Leonard, John |
| author_facet | Fu, Jiahui Du, Yilun Singh, Kurran Tenenbaum, Joshua Leonard, John |
| author_sort | Fu, Jiahui |
| collection | MIT |
| description | Robotics: Science and Systems Jul 10 – Jul 14, 2023 Daegu, Republic of Korea |
| first_indexed | 2024-09-23T14:13:30Z |
| format | Article |
| id | mit-1721.1/153745 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:13:30Z |
| publishDate | 2024 |
| publisher | Robotics: Science and Systems Foundation |
| record_format | dspace |
| spelling | mit-1721.1/1537452024-03-14T03:25:30Z NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects Fu, Jiahui Du, Yilun Singh, Kurran Tenenbaum, Joshua Leonard, John Robotics: Science and Systems Jul 10 – Jul 14, 2023 Daegu, Republic of Korea We present NeuSE, a novel Neural SE(3)-Equivariant Embedding for objects, and illustrate how it supports object SLAM for consistent spatial understanding with long-term scene changes. NeuSE is a set of latent object embeddings created from partial object observations. It serves as a compact point cloud surrogate for complete object models, encoding full shape information while transforming SE(3)-equivariantly in tandem with the object in the physical world. With NeuSE, relative frame transforms can be directly derived from inferred latent codes. Our proposed SLAM paradigm, using NeuSE for object shape and pose characterization, can operate independently or in conjunction with typical SLAM systems. It directly infers SE(3) camera pose constraints that are compatible with general SLAM pose graph optimization, while also maintaining a lightweight object-centric map that adapts to real-world changes. Our approach is evaluated on synthetic and real-world sequences featuring changed objects and shows improved localization accuracy and change-aware mapping capability, when working either standalone or jointly with a common SLAM pipeline. 2024-03-13T16:12:16Z 2024-03-13T16:12:16Z 2023-07-10 2024-03-13T15:41:00Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/153745 Fu, Jiahui, Du, Yilun, Singh, Kurran, Tenenbaum, Joshua and Leonard, John. 2023. "NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects." en 10.15607/rss.2023.xix.068 Creative Commons Attribution-Noncommercial-ShareAlike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Robotics: Science and Systems Foundation arxiv |
| spellingShingle | Fu, Jiahui Du, Yilun Singh, Kurran Tenenbaum, Joshua Leonard, John NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title | NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title_full | NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title_fullStr | NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title_full_unstemmed | NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title_short | NeuSE: Neural SE(3)-Equivariant Embedding for Consistent Spatial Understanding with Objects |
| title_sort | neuse neural se 3 equivariant embedding for consistent spatial understanding with objects |
| url | https://hdl.handle.net/1721.1/153745 |
| work_keys_str_mv | AT fujiahui neuseneuralse3equivariantembeddingforconsistentspatialunderstandingwithobjects AT duyilun neuseneuralse3equivariantembeddingforconsistentspatialunderstandingwithobjects AT singhkurran neuseneuralse3equivariantembeddingforconsistentspatialunderstandingwithobjects AT tenenbaumjoshua neuseneuralse3equivariantembeddingforconsistentspatialunderstandingwithobjects AT leonardjohn neuseneuralse3equivariantembeddingforconsistentspatialunderstandingwithobjects |