Variational relational point completion network for robust 3D classification
Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/172185 |
| _version_ | 1811695399362101248 |
|---|---|
| author | Pan, Liang Chen, Xinyi Cai, Zhongang Zhang, Junzhe Zhao, Haiyu Yi, Shuai Liu, Ziwei |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Pan, Liang Chen, Xinyi Cai, Zhongang Zhang, Junzhe Zhao, Haiyu Yi, Shuai Liu, Ziwei |
| author_sort | Pan, Liang |
| collection | NTU |
| description | Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40 dataset) containing over 200,000 high-quality scans, which render partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-the-art methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans. Moreover, we can achieve robust 3D classification for partial point clouds with the help of VRCNet, which can highly increase classification accuracy. |
| first_indexed | 2024-10-01T07:22:51Z |
| format | Journal Article |
| id | ntu-10356/172185 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:22:51Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1721852023-11-28T07:28:28Z Variational relational point completion network for robust 3D classification Pan, Liang Chen, Xinyi Cai, Zhongang Zhang, Junzhe Zhao, Haiyu Yi, Shuai Liu, Ziwei School of Computer Science and Engineering S-Lab Engineering::Computer science and engineering 3D Perception Point Cloud Completion Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40 dataset) containing over 200,000 high-quality scans, which render partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-the-art methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans. Moreover, we can achieve robust 3D classification for partial point clouds with the help of VRCNet, which can highly increase classification accuracy. Ministry of Education (MOE) Nanyang Technological University This work was supported in part by the Ministry of Education, Singapore, under its MOE AcRF Tier 2 (MOE-T2EP20221-0012), NTU NAP, and in part by the RIE2020 Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s). 2023-11-28T07:28:28Z 2023-11-28T07:28:28Z 2023 Journal Article Pan, L., Chen, X., Cai, Z., Zhang, J., Zhao, H., Yi, S. & Liu, Z. (2023). Variational relational point completion network for robust 3D classification. IEEE Transactions On Pattern Analysis and Machine Intelligence, 45(9), 11340-11351. https://dx.doi.org/10.1109/TPAMI.2023.3268305 0162-8828 https://hdl.handle.net/10356/172185 10.1109/TPAMI.2023.3268305 37083514 2-s2.0-85153801950 9 45 11340 11351 en MOE-T2EP20221-0012 IEEE Transactions on Pattern Analysis and Machine Intelligence © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering 3D Perception Point Cloud Completion Pan, Liang Chen, Xinyi Cai, Zhongang Zhang, Junzhe Zhao, Haiyu Yi, Shuai Liu, Ziwei Variational relational point completion network for robust 3D classification |
| title | Variational relational point completion network for robust 3D classification |
| title_full | Variational relational point completion network for robust 3D classification |
| title_fullStr | Variational relational point completion network for robust 3D classification |
| title_full_unstemmed | Variational relational point completion network for robust 3D classification |
| title_short | Variational relational point completion network for robust 3D classification |
| title_sort | variational relational point completion network for robust 3d classification |
| topic | Engineering::Computer science and engineering 3D Perception Point Cloud Completion |
| url | https://hdl.handle.net/10356/172185 |
| work_keys_str_mv | AT panliang variationalrelationalpointcompletionnetworkforrobust3dclassification AT chenxinyi variationalrelationalpointcompletionnetworkforrobust3dclassification AT caizhongang variationalrelationalpointcompletionnetworkforrobust3dclassification AT zhangjunzhe variationalrelationalpointcompletionnetworkforrobust3dclassification AT zhaohaiyu variationalrelationalpointcompletionnetworkforrobust3dclassification AT yishuai variationalrelationalpointcompletionnetworkforrobust3dclassification AT liuziwei variationalrelationalpointcompletionnetworkforrobust3dclassification |