Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation
Deep learning approaches to estimating full 3D orientations of objects, in addition to object classes, are limited in their accuracies, due to the difficulty in learning the continuous nature of three-axis orientation variations by regression or classification with sufficient generalization. This pa...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/18/6108 |
| _version_ | 1797517308432744448 |
|---|---|
| author | Sukhan Lee Yongjun Yang |
| author_facet | Sukhan Lee Yongjun Yang |
| author_sort | Sukhan Lee |
| collection | DOAJ |
| description | Deep learning approaches to estimating full 3D orientations of objects, in addition to object classes, are limited in their accuracies, due to the difficulty in learning the continuous nature of three-axis orientation variations by regression or classification with sufficient generalization. This paper presents a novel progressive deep learning framework, herein referred to as 3D POCO Net, that offers high accuracy in estimating orientations about three rotational axes yet with efficiency in network complexity. The proposed 3D POCO Net is configured, using four PointNet-based networks for independently representing the object class and three individual axes of rotations. The four independent networks are linked by in-between association subnetworks that are trained to progressively map the global features learned by individual networks one after another for fine-tuning the independent networks. In 3D POCO Net, high accuracy is achieved by combining a high precision classification based on a large number of orientation classes with a regression based on a weighted sum of classification outputs, while high efficiency is maintained by a progressive framework by which a large number of orientation classes are grouped into independent networks linked by association subnetworks. We implemented 3D POCO Net for full three-axis orientation variations and trained it with about 146 million orientation variations augmented from the ModelNet10 dataset. The testing results show that we can achieve an orientation regression error of about 2.5° with about 90% accuracy in object classification for general three-axis orientation estimation and object classification. Furthermore, we demonstrate that a pre-trained 3D POCO Net can serve as an orientation representation platform based on which orientations as well as object classes of partial point clouds from occluded objects are learned in the form of transfer learning. |
| first_indexed | 2024-03-10T07:13:51Z |
| format | Article |
| id | doaj.art-d306fc51ed9747f784601e75e1f46674 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T07:13:51Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d306fc51ed9747f784601e75e1f466742023-11-22T15:11:45ZengMDPI AGSensors1424-82202021-09-012118610810.3390/s21186108Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud RepresentationSukhan Lee0Yongjun Yang1Artificial Intelligence Department, Sungkyunkwan University, Suwon 16419, KoreaSchool of Information & Communication Engineering, Sungkyunkwan University, Suwon 16419, KoreaDeep learning approaches to estimating full 3D orientations of objects, in addition to object classes, are limited in their accuracies, due to the difficulty in learning the continuous nature of three-axis orientation variations by regression or classification with sufficient generalization. This paper presents a novel progressive deep learning framework, herein referred to as 3D POCO Net, that offers high accuracy in estimating orientations about three rotational axes yet with efficiency in network complexity. The proposed 3D POCO Net is configured, using four PointNet-based networks for independently representing the object class and three individual axes of rotations. The four independent networks are linked by in-between association subnetworks that are trained to progressively map the global features learned by individual networks one after another for fine-tuning the independent networks. In 3D POCO Net, high accuracy is achieved by combining a high precision classification based on a large number of orientation classes with a regression based on a weighted sum of classification outputs, while high efficiency is maintained by a progressive framework by which a large number of orientation classes are grouped into independent networks linked by association subnetworks. We implemented 3D POCO Net for full three-axis orientation variations and trained it with about 146 million orientation variations augmented from the ModelNet10 dataset. The testing results show that we can achieve an orientation regression error of about 2.5° with about 90% accuracy in object classification for general three-axis orientation estimation and object classification. Furthermore, we demonstrate that a pre-trained 3D POCO Net can serve as an orientation representation platform based on which orientations as well as object classes of partial point clouds from occluded objects are learned in the form of transfer learning.https://www.mdpi.com/1424-8220/21/18/6108orientation representation3D point cloud3D objectprogressive learningassociation network |
| spellingShingle | Sukhan Lee Yongjun Yang Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation Sensors orientation representation 3D point cloud 3D object progressive learning association network |
| title | Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation |
| title_full | Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation |
| title_fullStr | Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation |
| title_full_unstemmed | Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation |
| title_short | Progressive Deep Learning Framework for Recognizing 3D Orientations and Object Class Based on Point Cloud Representation |
| title_sort | progressive deep learning framework for recognizing 3d orientations and object class based on point cloud representation |
| topic | orientation representation 3D point cloud 3D object progressive learning association network |
| url | https://www.mdpi.com/1424-8220/21/18/6108 |
| work_keys_str_mv | AT sukhanlee progressivedeeplearningframeworkforrecognizing3dorientationsandobjectclassbasedonpointcloudrepresentation AT yongjunyang progressivedeeplearningframeworkforrecognizing3dorientationsandobjectclassbasedonpointcloudrepresentation |