Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy
Skeleton-based Graph Convolutional Networks (GCN) for human action and interaction recognition have received considerable attention of researchers due to its compact and view-invariant nature of skeleton data. However, the static skeleton graph topology in conventional GCNs does not reflect the impl...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9749063/ |
| _version_ | 1817991321908936704 |
|---|---|
| author | Quanyu Wang Kaixiang Zhang Manjotho Ali Asghar |
| author_facet | Quanyu Wang Kaixiang Zhang Manjotho Ali Asghar |
| author_sort | Quanyu Wang |
| collection | DOAJ |
| description | Skeleton-based Graph Convolutional Networks (GCN) for human action and interaction recognition have received considerable attention of researchers due to its compact and view-invariant nature of skeleton data. However, the static skeleton graph topology in conventional GCNs does not reflect the implicit relationships of non-adjacent joints, which contain vital latent information for a skeleton pose in an action sequence. Moreover, traditional tri-categorical node partitioning strategy discards much of the motion dependencies along temporal dimension for non-physically connected edges. We propose an extended skeleton graph topology along with extended partitioning strategy to extract much of the non-adjacent joint relational information in the model for robust discriminative features. Extended skeleton graph represents joints as vertices and weighted edges represent intrinsic and extrinsic relationships between physically connected and non-physically connected joints respectively. Furthermore, extended partitioning strategy divides the input graph for GCN as five-categorical fixed-length tensor to encompass maximal motion dependencies. Finally, the extended skeleton graph and partitioning strategy are realized by adopting Spatio-Temporal Graph Convolutional Network (ST-GCN). The experiments carried out over three large scale datasets NTU-RGB+D, NTU-RGB+D 120 and Kinetics-Skeleton show improved performance over conventional state-of-the-art ST-GCNs. |
| first_indexed | 2024-04-14T01:10:52Z |
| format | Article |
| id | doaj.art-0a4f8d5ae18b49a7978f538aa2f69add |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-14T01:10:52Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-0a4f8d5ae18b49a7978f538aa2f69add2022-12-22T02:21:02ZengIEEEIEEE Access2169-35362022-01-0110414034141010.1109/ACCESS.2022.31647119749063Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning StrategyQuanyu Wang0Kaixiang Zhang1https://orcid.org/0000-0002-1285-3866Manjotho Ali Asghar2School of Computer Science and Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Computer Science and Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Computer Science and Technology, Beijing Institute of Technology, Beijing, ChinaSkeleton-based Graph Convolutional Networks (GCN) for human action and interaction recognition have received considerable attention of researchers due to its compact and view-invariant nature of skeleton data. However, the static skeleton graph topology in conventional GCNs does not reflect the implicit relationships of non-adjacent joints, which contain vital latent information for a skeleton pose in an action sequence. Moreover, traditional tri-categorical node partitioning strategy discards much of the motion dependencies along temporal dimension for non-physically connected edges. We propose an extended skeleton graph topology along with extended partitioning strategy to extract much of the non-adjacent joint relational information in the model for robust discriminative features. Extended skeleton graph represents joints as vertices and weighted edges represent intrinsic and extrinsic relationships between physically connected and non-physically connected joints respectively. Furthermore, extended partitioning strategy divides the input graph for GCN as five-categorical fixed-length tensor to encompass maximal motion dependencies. Finally, the extended skeleton graph and partitioning strategy are realized by adopting Spatio-Temporal Graph Convolutional Network (ST-GCN). The experiments carried out over three large scale datasets NTU-RGB+D, NTU-RGB+D 120 and Kinetics-Skeleton show improved performance over conventional state-of-the-art ST-GCNs.https://ieeexplore.ieee.org/document/9749063/Action recognitiondeep learninggraph convolutional networkhuman skeletonpartition strategy |
| spellingShingle | Quanyu Wang Kaixiang Zhang Manjotho Ali Asghar Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy IEEE Access Action recognition deep learning graph convolutional network human skeleton partition strategy |
| title | Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy |
| title_full | Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy |
| title_fullStr | Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy |
| title_full_unstemmed | Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy |
| title_short | Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy |
| title_sort | skeleton based st gcn for human action recognition with extended skeleton graph and partitioning strategy |
| topic | Action recognition deep learning graph convolutional network human skeleton partition strategy |
| url | https://ieeexplore.ieee.org/document/9749063/ |
| work_keys_str_mv | AT quanyuwang skeletonbasedstgcnforhumanactionrecognitionwithextendedskeletongraphandpartitioningstrategy AT kaixiangzhang skeletonbasedstgcnforhumanactionrecognitionwithextendedskeletongraphandpartitioningstrategy AT manjothoaliasghar skeletonbasedstgcnforhumanactionrecognitionwithextendedskeletongraphandpartitioningstrategy |