A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning
Learning the relationship between the part and whole of an object, such as humans recognizing objects, is a challenging task. In this paper, we specifically design a novel neural network to explore the local-to-global cognition of 3D models and the aggregation of structural contextual features in 3D...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-05-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/5/678 |
| _version_ | 1797500022301917184 |
|---|---|
| author | Yu Chen Jieyu Zhao Qilu Qiu |
| author_facet | Yu Chen Jieyu Zhao Qilu Qiu |
| author_sort | Yu Chen |
| collection | DOAJ |
| description | Learning the relationship between the part and whole of an object, such as humans recognizing objects, is a challenging task. In this paper, we specifically design a novel neural network to explore the local-to-global cognition of 3D models and the aggregation of structural contextual features in 3D space, inspired by the recent success of Transformer in natural language processing (NLP) and impressive strides in image analysis tasks such as image classification and object detection. We build a 3D shape Transformer based on local shape representation, which provides relation learning between local patches on 3D mesh models. Similar to token (word) states in NLP, we propose local shape tokens to encode local geometric information. On this basis, we design a shape-Transformer-based capsule routing algorithm. By applying an iterative capsule routing algorithm, local shape information can be further aggregated into high-level capsules containing deeper contextual information so as to realize the cognition from the local to the whole. We performed classification tasks on the deformable 3D object data sets SHREC10 and SHREC15 and the large data set ModelNet40, and obtained profound results, which shows that our model has excellent performance in complex 3D model recognition and big data feature learning. |
| first_indexed | 2024-03-10T03:55:51Z |
| format | Article |
| id | doaj.art-2deee015cdc94c9f8620bfef6fefaf51 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T03:55:51Z |
| publishDate | 2022-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-2deee015cdc94c9f8620bfef6fefaf512023-11-23T10:55:34ZengMDPI AGEntropy1099-43002022-05-0124567810.3390/e24050678A Transformer-Based Capsule Network for 3D Part–Whole Relationship LearningYu Chen0Jieyu Zhao1Qilu Qiu2Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, ChinaFaculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, ChinaFaculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, ChinaLearning the relationship between the part and whole of an object, such as humans recognizing objects, is a challenging task. In this paper, we specifically design a novel neural network to explore the local-to-global cognition of 3D models and the aggregation of structural contextual features in 3D space, inspired by the recent success of Transformer in natural language processing (NLP) and impressive strides in image analysis tasks such as image classification and object detection. We build a 3D shape Transformer based on local shape representation, which provides relation learning between local patches on 3D mesh models. Similar to token (word) states in NLP, we propose local shape tokens to encode local geometric information. On this basis, we design a shape-Transformer-based capsule routing algorithm. By applying an iterative capsule routing algorithm, local shape information can be further aggregated into high-level capsules containing deeper contextual information so as to realize the cognition from the local to the whole. We performed classification tasks on the deformable 3D object data sets SHREC10 and SHREC15 and the large data set ModelNet40, and obtained profound results, which shows that our model has excellent performance in complex 3D model recognition and big data feature learning.https://www.mdpi.com/1099-4300/24/5/6783D shape transformerlocal-to-global cognitionshape-Transformer-based capsuledeformable 3D object |
| spellingShingle | Yu Chen Jieyu Zhao Qilu Qiu A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning Entropy 3D shape transformer local-to-global cognition shape-Transformer-based capsule deformable 3D object |
| title | A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning |
| title_full | A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning |
| title_fullStr | A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning |
| title_full_unstemmed | A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning |
| title_short | A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning |
| title_sort | transformer based capsule network for 3d part whole relationship learning |
| topic | 3D shape transformer local-to-global cognition shape-Transformer-based capsule deformable 3D object |
| url | https://www.mdpi.com/1099-4300/24/5/678 |
| work_keys_str_mv | AT yuchen atransformerbasedcapsulenetworkfor3dpartwholerelationshiplearning AT jieyuzhao atransformerbasedcapsulenetworkfor3dpartwholerelationshiplearning AT qiluqiu atransformerbasedcapsulenetworkfor3dpartwholerelationshiplearning AT yuchen transformerbasedcapsulenetworkfor3dpartwholerelationshiplearning AT jieyuzhao transformerbasedcapsulenetworkfor3dpartwholerelationshiplearning AT qiluqiu transformerbasedcapsulenetworkfor3dpartwholerelationshiplearning |