Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation
Semantic feature learning on 3D point clouds is quite challenging because of their irregular and unordered data structure. In this paper, we propose a novel structure-aware convolution (SAC) to generalize deep learning on regular grids to irregular 3D point clouds. Similar to the template-matching p...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-02-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/4/634 |
| _version_ | 1818973909043642368 |
|---|---|
| author | Lei Wang Yuxuan Liu Shenman Zhang Jixing Yan Pengjie Tao |
| author_facet | Lei Wang Yuxuan Liu Shenman Zhang Jixing Yan Pengjie Tao |
| author_sort | Lei Wang |
| collection | DOAJ |
| description | Semantic feature learning on 3D point clouds is quite challenging because of their irregular and unordered data structure. In this paper, we propose a novel structure-aware convolution (SAC) to generalize deep learning on regular grids to irregular 3D point clouds. Similar to the template-matching process of convolution on 2D images, the key of our SAC is to match the point clouds’ neighborhoods with a series of 3D kernels, where each kernel can be regarded as a “geometric template” formed by a set of learnable 3D points. Thus, the interested geometric structures of the input point clouds can be activated by the corresponding kernels. To verify the effectiveness of the proposed SAC, we embedded it into three recently developed point cloud deep learning networks (PointNet, PointNet++, and KCNet) as a lightweight module, and evaluated its performance on both classification and segmentation tasks. Experimental results show that, benefiting from the geometric structure learning capability of our SAC, all these back-end networks achieved better classification and segmentation performance (e.g., +2.77% mean accuracy for classification and +4.99% mean intersection over union (IoU) for segmentation) with few additional parameters. Furthermore, results also demonstrate that the proposed SAC is helpful in improving the robustness of networks with the constraints of geometric structures. |
| first_indexed | 2024-12-20T15:31:39Z |
| format | Article |
| id | doaj.art-7992453f070c4c60a6e1e84bbb20e3f1 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-12-20T15:31:39Z |
| publishDate | 2020-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-7992453f070c4c60a6e1e84bbb20e3f12022-12-21T19:35:34ZengMDPI AGRemote Sensing2072-42922020-02-0112463410.3390/rs12040634rs12040634Structure-Aware Convolution for 3D Point Cloud Classification and SegmentationLei Wang0Yuxuan Liu1Shenman Zhang2Jixing Yan3Pengjie Tao4State Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSchool of Highway, Chang’an University, Xi’an 710064, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, ChinaSemantic feature learning on 3D point clouds is quite challenging because of their irregular and unordered data structure. In this paper, we propose a novel structure-aware convolution (SAC) to generalize deep learning on regular grids to irregular 3D point clouds. Similar to the template-matching process of convolution on 2D images, the key of our SAC is to match the point clouds’ neighborhoods with a series of 3D kernels, where each kernel can be regarded as a “geometric template” formed by a set of learnable 3D points. Thus, the interested geometric structures of the input point clouds can be activated by the corresponding kernels. To verify the effectiveness of the proposed SAC, we embedded it into three recently developed point cloud deep learning networks (PointNet, PointNet++, and KCNet) as a lightweight module, and evaluated its performance on both classification and segmentation tasks. Experimental results show that, benefiting from the geometric structure learning capability of our SAC, all these back-end networks achieved better classification and segmentation performance (e.g., +2.77% mean accuracy for classification and +4.99% mean intersection over union (IoU) for segmentation) with few additional parameters. Furthermore, results also demonstrate that the proposed SAC is helpful in improving the robustness of networks with the constraints of geometric structures.https://www.mdpi.com/2072-4292/12/4/634structure-aware convolution3d point cloudclassificationsegmentation |
| spellingShingle | Lei Wang Yuxuan Liu Shenman Zhang Jixing Yan Pengjie Tao Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation Remote Sensing structure-aware convolution 3d point cloud classification segmentation |
| title | Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation |
| title_full | Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation |
| title_fullStr | Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation |
| title_full_unstemmed | Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation |
| title_short | Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation |
| title_sort | structure aware convolution for 3d point cloud classification and segmentation |
| topic | structure-aware convolution 3d point cloud classification segmentation |
| url | https://www.mdpi.com/2072-4292/12/4/634 |
| work_keys_str_mv | AT leiwang structureawareconvolutionfor3dpointcloudclassificationandsegmentation AT yuxuanliu structureawareconvolutionfor3dpointcloudclassificationandsegmentation AT shenmanzhang structureawareconvolutionfor3dpointcloudclassificationandsegmentation AT jixingyan structureawareconvolutionfor3dpointcloudclassificationandsegmentation AT pengjietao structureawareconvolutionfor3dpointcloudclassificationandsegmentation |