Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation
Continuous implicit representations can flexibly describe complex 3D geometry and offer excellent potential for 3D point cloud analysis. However, it remains challenging for existing point-based deep learning architectures to leverage the implicit representations due to the discrepancy in data struct...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/1/61 |
| _version_ | 1797439800234475520 |
|---|---|
| author | Zexin Yang Qin Ye Jantien Stoter Liangliang Nan |
| author_facet | Zexin Yang Qin Ye Jantien Stoter Liangliang Nan |
| author_sort | Zexin Yang |
| collection | DOAJ |
| description | Continuous implicit representations can flexibly describe complex 3D geometry and offer excellent potential for 3D point cloud analysis. However, it remains challenging for existing point-based deep learning architectures to leverage the implicit representations due to the discrepancy in data structures between implicit fields and point clouds. In this work, we propose a new point cloud representation by integrating the 3D Cartesian coordinates with the intrinsic geometric information encapsulated in its implicit field. Specifically, we parameterize the continuous unsigned distance field around each point into a low-dimensional feature vector that captures the local geometry. Then we concatenate the 3D Cartesian coordinates of each point with its encoded implicit feature vector as the network input. The proposed method can be plugged into an existing network architecture as a module without trainable weights. We also introduce a novel local canonicalization approach to ensure the transformation-invariance of encoded implicit features. With its local mechanism, our implicit feature encoding module can be applied to not only point clouds of single objects but also those of complex real-world scenes. We have validated the effectiveness of our approach using five well-known point-based deep networks (i.e., PointNet, SuperPoint Graph, RandLA-Net, CurveNet, and Point Structuring Net) on object-level classification and scene-level semantic segmentation tasks. Extensive experiments on both synthetic and real-world datasets have demonstrated the effectiveness of the proposed point representation. |
| first_indexed | 2024-03-09T11:59:22Z |
| format | Article |
| id | doaj.art-4182d76fed42445d8cf0e8b83463bad0 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T11:59:22Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-4182d76fed42445d8cf0e8b83463bad02023-11-30T23:05:13ZengMDPI AGRemote Sensing2072-42922022-12-011516110.3390/rs15010061Enriching Point Clouds with Implicit Representations for 3D Classification and SegmentationZexin Yang0Qin Ye1Jantien Stoter2Liangliang Nan3College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, ChinaCollege of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China3D Geoinformation Research Group, Delft University of Technology, 2628 BL Delft, The Netherlands3D Geoinformation Research Group, Delft University of Technology, 2628 BL Delft, The NetherlandsContinuous implicit representations can flexibly describe complex 3D geometry and offer excellent potential for 3D point cloud analysis. However, it remains challenging for existing point-based deep learning architectures to leverage the implicit representations due to the discrepancy in data structures between implicit fields and point clouds. In this work, we propose a new point cloud representation by integrating the 3D Cartesian coordinates with the intrinsic geometric information encapsulated in its implicit field. Specifically, we parameterize the continuous unsigned distance field around each point into a low-dimensional feature vector that captures the local geometry. Then we concatenate the 3D Cartesian coordinates of each point with its encoded implicit feature vector as the network input. The proposed method can be plugged into an existing network architecture as a module without trainable weights. We also introduce a novel local canonicalization approach to ensure the transformation-invariance of encoded implicit features. With its local mechanism, our implicit feature encoding module can be applied to not only point clouds of single objects but also those of complex real-world scenes. We have validated the effectiveness of our approach using five well-known point-based deep networks (i.e., PointNet, SuperPoint Graph, RandLA-Net, CurveNet, and Point Structuring Net) on object-level classification and scene-level semantic segmentation tasks. Extensive experiments on both synthetic and real-world datasets have demonstrated the effectiveness of the proposed point representation.https://www.mdpi.com/2072-4292/15/1/61point cloudsemantic segmentationobject classificationimplicit representation |
| spellingShingle | Zexin Yang Qin Ye Jantien Stoter Liangliang Nan Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation Remote Sensing point cloud semantic segmentation object classification implicit representation |
| title | Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation |
| title_full | Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation |
| title_fullStr | Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation |
| title_full_unstemmed | Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation |
| title_short | Enriching Point Clouds with Implicit Representations for 3D Classification and Segmentation |
| title_sort | enriching point clouds with implicit representations for 3d classification and segmentation |
| topic | point cloud semantic segmentation object classification implicit representation |
| url | https://www.mdpi.com/2072-4292/15/1/61 |
| work_keys_str_mv | AT zexinyang enrichingpointcloudswithimplicitrepresentationsfor3dclassificationandsegmentation AT qinye enrichingpointcloudswithimplicitrepresentationsfor3dclassificationandsegmentation AT jantienstoter enrichingpointcloudswithimplicitrepresentationsfor3dclassificationandsegmentation AT liangliangnan enrichingpointcloudswithimplicitrepresentationsfor3dclassificationandsegmentation |