Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network
Compared with the traditional RGB-based methods, the skeleton-based action recognition methods have become the main research direction in the field of computer vision in recent years because they are less affected by many factors such as illumination, viewing angle and background complexity. However...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
2021-04-01
|
| Series: | Jisuanji kexue yu tansuo |
| Subjects: | |
| Online Access: | http://fcst.ceaj.org/CN/abstract/abstract2658.shtml |
| _version_ | 1828916841091694592 |
|---|---|
| author | SU Jiangyi, SONG Xiaoning, WU Xiaojun, YU Dongjun |
| author_facet | SU Jiangyi, SONG Xiaoning, WU Xiaojun, YU Dongjun |
| author_sort | SU Jiangyi, SONG Xiaoning, WU Xiaojun, YU Dongjun |
| collection | DOAJ |
| description | Compared with the traditional RGB-based methods, the skeleton-based action recognition methods have become the main research direction in the field of computer vision in recent years because they are less affected by many factors such as illumination, viewing angle and background complexity. However, the current skeleton-based methods still have some problems such as large parameters, long time-consuming and high computational complexity, which makes it complicated and difficult to meet the requirements of efficiency and accuracy simultaneously. To address these issues, a lightweight graph convolution network using multi-modal data fusion is proposed. Firstly, the multi-modal information flow data are fused by multi-modal data fusion method. Secondly, the spatial and temporal information of human joints are extracted using spatial and temporal modules respectively. Finally, the classification results are obtained through the fully connected layer. Experimental results conducted on the two commonly used datasets including NTU60 RGB+D and NTU120 RGB+D demonstrate that the proposed network outperforms some mainstream methods in the last two years in both recognition accuracy and efficiency, thus verifying that the network has excellent performance in terms of accuracy, while considering time efficiency and computational cost. |
| first_indexed | 2024-12-13T20:40:53Z |
| format | Article |
| id | doaj.art-17f6bac1026b4ac0b9d0d72221daad66 |
| institution | Directory Open Access Journal |
| issn | 1673-9418 |
| language | zho |
| last_indexed | 2024-12-13T20:40:53Z |
| publishDate | 2021-04-01 |
| publisher | Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press |
| record_format | Article |
| series | Jisuanji kexue yu tansuo |
| spelling | doaj.art-17f6bac1026b4ac0b9d0d72221daad662022-12-21T23:32:09ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182021-04-0115473374210.3778/j.issn.1673-9418.2008051Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional NetworkSU Jiangyi, SONG Xiaoning, WU Xiaojun, YU Dongjun01. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China 2. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, ChinaCompared with the traditional RGB-based methods, the skeleton-based action recognition methods have become the main research direction in the field of computer vision in recent years because they are less affected by many factors such as illumination, viewing angle and background complexity. However, the current skeleton-based methods still have some problems such as large parameters, long time-consuming and high computational complexity, which makes it complicated and difficult to meet the requirements of efficiency and accuracy simultaneously. To address these issues, a lightweight graph convolution network using multi-modal data fusion is proposed. Firstly, the multi-modal information flow data are fused by multi-modal data fusion method. Secondly, the spatial and temporal information of human joints are extracted using spatial and temporal modules respectively. Finally, the classification results are obtained through the fully connected layer. Experimental results conducted on the two commonly used datasets including NTU60 RGB+D and NTU120 RGB+D demonstrate that the proposed network outperforms some mainstream methods in the last two years in both recognition accuracy and efficiency, thus verifying that the network has excellent performance in terms of accuracy, while considering time efficiency and computational cost.http://fcst.ceaj.org/CN/abstract/abstract2658.shtmlaction recognitionhuman skeletonlightweightgraph convolutional network |
| spellingShingle | SU Jiangyi, SONG Xiaoning, WU Xiaojun, YU Dongjun Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network Jisuanji kexue yu tansuo action recognition human skeleton lightweight graph convolutional network |
| title | Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network |
| title_full | Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network |
| title_fullStr | Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network |
| title_full_unstemmed | Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network |
| title_short | Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network |
| title_sort | skeleton based action recognition algorithm on multi modal lightweight graph convolutional network |
| topic | action recognition human skeleton lightweight graph convolutional network |
| url | http://fcst.ceaj.org/CN/abstract/abstract2658.shtml |
| work_keys_str_mv | AT sujiangyisongxiaoningwuxiaojunyudongjun skeletonbasedactionrecognitionalgorithmonmultimodallightweightgraphconvolutionalnetwork |