Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition
In recent years, skeleton-based action recognition, graph convolutional networks, have achieved remarkable performance. In these existing works, the features of all nodes in the neighbor set are aggregated into the updated features of the root node, while these features are located in the same featu...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9316772/ |
| _version_ | 1819175906154905600 |
|---|---|
| author | Jun Xie Wentian Xin Ruyi Liu Lijie Sheng Xiangzeng Liu Xuesong Gao Sheng Zhong Lei Tang Qiguang Miao |
| author_facet | Jun Xie Wentian Xin Ruyi Liu Lijie Sheng Xiangzeng Liu Xuesong Gao Sheng Zhong Lei Tang Qiguang Miao |
| author_sort | Jun Xie |
| collection | DOAJ |
| description | In recent years, skeleton-based action recognition, graph convolutional networks, have achieved remarkable performance. In these existing works, the features of all nodes in the neighbor set are aggregated into the updated features of the root node, while these features are located in the same feature channel determined by the same 1 × 1 convolution filter. This may not be optimal for capturing the features of spatial dimensions among adjacent vertices effectively. Besides, the effect of feature channels that are independent of the current action on the performance of the model is rarely investigated in existing methods. In this paper, we propose cross-channel graph convolutional networks for skeleton-based action recognition. The features fusion mechanism in our network is cross-channel, i.e, the updated feature of the root node is derived from different feature channels. Because different feature channels come from different 1 × 1 convolution filters, the cross-channel fusion mechanism significantly improves the ability of the model to capture local features among adjacent vertices. Moreover, by introducing a channel attention mechanism to our model, we suppress the influence of feature channels unrelated to action recognition on model performance, which improves the robustness of the model against the feature channels independent of the current action. Extensive experiments on the two large-scale datasets, NTU-RGB+D and KineticsSkeleton, demonstrate that the performance of our model exceeds the current mainstream methods. |
| first_indexed | 2024-12-22T21:02:18Z |
| format | Article |
| id | doaj.art-e69a6429498f4b65a985254565a5f841 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T21:02:18Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e69a6429498f4b65a985254565a5f8412022-12-21T18:12:47ZengIEEEIEEE Access2169-35362021-01-0199055906510.1109/ACCESS.2021.30498089316772Cross-Channel Graph Convolutional Networks for Skeleton-Based Action RecognitionJun Xie0https://orcid.org/0000-0002-4359-1045Wentian Xin1https://orcid.org/0000-0002-6479-3092Ruyi Liu2https://orcid.org/0000-0001-9231-1081Lijie Sheng3Xiangzeng Liu4https://orcid.org/0000-0002-2751-6096Xuesong Gao5Sheng Zhong6Lei Tang7Qiguang Miao8https://orcid.org/0000-0001-6766-8310School of Computer Science and Technology, Xidian University, Xi’an, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an, ChinaState Key Laboratory of Digital Multimedia Technology, Hisense Company Ltd., Qingdao, ChinaSchool of Information Science and Technology, Northwest University of China, Xi’an, ChinaXi’an Microelectronics Technology Institute, Xi’an, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an, ChinaIn recent years, skeleton-based action recognition, graph convolutional networks, have achieved remarkable performance. In these existing works, the features of all nodes in the neighbor set are aggregated into the updated features of the root node, while these features are located in the same feature channel determined by the same 1 × 1 convolution filter. This may not be optimal for capturing the features of spatial dimensions among adjacent vertices effectively. Besides, the effect of feature channels that are independent of the current action on the performance of the model is rarely investigated in existing methods. In this paper, we propose cross-channel graph convolutional networks for skeleton-based action recognition. The features fusion mechanism in our network is cross-channel, i.e, the updated feature of the root node is derived from different feature channels. Because different feature channels come from different 1 × 1 convolution filters, the cross-channel fusion mechanism significantly improves the ability of the model to capture local features among adjacent vertices. Moreover, by introducing a channel attention mechanism to our model, we suppress the influence of feature channels unrelated to action recognition on model performance, which improves the robustness of the model against the feature channels independent of the current action. Extensive experiments on the two large-scale datasets, NTU-RGB+D and KineticsSkeleton, demonstrate that the performance of our model exceeds the current mainstream methods.https://ieeexplore.ieee.org/document/9316772/Skeleton-based action recognitiongraph convolutional networkchannel attentionaction recognition |
| spellingShingle | Jun Xie Wentian Xin Ruyi Liu Lijie Sheng Xiangzeng Liu Xuesong Gao Sheng Zhong Lei Tang Qiguang Miao Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition IEEE Access Skeleton-based action recognition graph convolutional network channel attention action recognition |
| title | Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_full | Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_fullStr | Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_full_unstemmed | Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_short | Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_sort | cross channel graph convolutional networks for skeleton based action recognition |
| topic | Skeleton-based action recognition graph convolutional network channel attention action recognition |
| url | https://ieeexplore.ieee.org/document/9316772/ |
| work_keys_str_mv | AT junxie crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT wentianxin crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT ruyiliu crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT lijiesheng crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT xiangzengliu crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT xuesonggao crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT shengzhong crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT leitang crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition AT qiguangmiao crosschannelgraphconvolutionalnetworksforskeletonbasedactionrecognition |