Spatial–temporal graph neural network based on node attention
Recently, the method of using graph neural network based on skeletons for action recognition has become more and more popular, due to the fact that a skeleton can carry very intuitive and rich action information, without being affected by background, light and other factors. The spatial–temporal gra...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Sciendo
2022-04-01
|
| Series: | Applied Mathematics and Nonlinear Sciences |
| Subjects: | |
| Online Access: | https://doi.org/10.2478/amns.2022.1.00005 |
| _version_ | 1797258375438794752 |
|---|---|
| author | Li Qiang Wan Jun Zhang Wucong Kweh Qian Long |
| author_facet | Li Qiang Wan Jun Zhang Wucong Kweh Qian Long |
| author_sort | Li Qiang |
| collection | DOAJ |
| description | Recently, the method of using graph neural network based on skeletons for action recognition has become more and more popular, due to the fact that a skeleton can carry very intuitive and rich action information, without being affected by background, light and other factors. The spatial–temporal graph convolutional neural network (ST-GCN) is a dynamic skeleton model that automatically learns spatial–temporal model from data, which not only has stronger expression ability, but also has stronger generalisation ability, showing remarkable results on public data sets. However, the ST-GCN network directly learns the information of adjacent nodes (local information), and is insufficient in learning the relations of non-adjacent nodes (global information), such as clapping action that requires learning the related information of non-adjacent nodes. Therefore, this paper proposes an ST-GCN based on node attention (NA-STGCN), so as to solve the problem of insufficient global information in ST-GCN by introducing node attention module to explicitly model the interdependence between global nodes. The experimental results on the NTU-RGB+D set show that the node attention module can effectively improve the accuracy and feature representation ability of the existing algorithms, and obviously improve the recognition effect of the actions that need global information. |
| first_indexed | 2024-03-13T04:40:18Z |
| format | Article |
| id | doaj.art-60b4d27c9b7f484fad073fda4783fc98 |
| institution | Directory Open Access Journal |
| issn | 2444-8656 |
| language | English |
| last_indexed | 2024-04-24T22:52:32Z |
| publishDate | 2022-04-01 |
| publisher | Sciendo |
| record_format | Article |
| series | Applied Mathematics and Nonlinear Sciences |
| spelling | doaj.art-60b4d27c9b7f484fad073fda4783fc982024-03-18T10:29:01ZengSciendoApplied Mathematics and Nonlinear Sciences2444-86562022-04-017270371210.2478/amns.2022.1.00005Spatial–temporal graph neural network based on node attentionLi Qiang0Wan Jun1Zhang Wucong2Kweh Qian Long3School of Electronic and Information Engineering, South China University of Technology, Guangzhou, ChinaMidea Intelligent Life Research Institute, Midea Real Estate Holding Limited, Foshan, ChinaMidea Intelligent Life Research Institute, Midea Real Estate Holding Limited, Foshan, ChinaCanadian University DubaiRecently, the method of using graph neural network based on skeletons for action recognition has become more and more popular, due to the fact that a skeleton can carry very intuitive and rich action information, without being affected by background, light and other factors. The spatial–temporal graph convolutional neural network (ST-GCN) is a dynamic skeleton model that automatically learns spatial–temporal model from data, which not only has stronger expression ability, but also has stronger generalisation ability, showing remarkable results on public data sets. However, the ST-GCN network directly learns the information of adjacent nodes (local information), and is insufficient in learning the relations of non-adjacent nodes (global information), such as clapping action that requires learning the related information of non-adjacent nodes. Therefore, this paper proposes an ST-GCN based on node attention (NA-STGCN), so as to solve the problem of insufficient global information in ST-GCN by introducing node attention module to explicitly model the interdependence between global nodes. The experimental results on the NTU-RGB+D set show that the node attention module can effectively improve the accuracy and feature representation ability of the existing algorithms, and obviously improve the recognition effect of the actions that need global information.https://doi.org/10.2478/amns.2022.1.00005action recognitionskeletonsspatial–temporal graph convolutionattention mechanism |
| spellingShingle | Li Qiang Wan Jun Zhang Wucong Kweh Qian Long Spatial–temporal graph neural network based on node attention Applied Mathematics and Nonlinear Sciences action recognition skeletons spatial–temporal graph convolution attention mechanism |
| title | Spatial–temporal graph neural network based on node attention |
| title_full | Spatial–temporal graph neural network based on node attention |
| title_fullStr | Spatial–temporal graph neural network based on node attention |
| title_full_unstemmed | Spatial–temporal graph neural network based on node attention |
| title_short | Spatial–temporal graph neural network based on node attention |
| title_sort | spatial temporal graph neural network based on node attention |
| topic | action recognition skeletons spatial–temporal graph convolution attention mechanism |
| url | https://doi.org/10.2478/amns.2022.1.00005 |
| work_keys_str_mv | AT liqiang spatialtemporalgraphneuralnetworkbasedonnodeattention AT wanjun spatialtemporalgraphneuralnetworkbasedonnodeattention AT zhangwucong spatialtemporalgraphneuralnetworkbasedonnodeattention AT kwehqianlong spatialtemporalgraphneuralnetworkbasedonnodeattention |