Graph convolution network based skeleton action recognition with DCT features
Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Grap...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/172751 |
| _version_ | 1811683357909581824 |
|---|---|
| author | Hei, Hao |
| author2 | Alex Chichung Kot |
| author_facet | Alex Chichung Kot Hei, Hao |
| author_sort | Hei, Hao |
| collection | NTU |
| description | Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Graph Convolutional Networks (GCNs) on skeleton modality, little attention has been paid to taking advantage of the frequency representation of skeleton data. In this project, our objective is to study the effect of utilizing skeleton features in the frequency domain to perform HAR with GCN. To achieve
the target, we first conduct a thorough review of current approaches for HAR and frequency analysis. Inspired by research on attention mechanism, we proposed to combine channel attention and 2-D Discrete Cosine Transform (DCT) as a universal layer of a deep learning network to utilize the frequency information from skeleton data, which can be inserted in the current GCNs for improvements in classification accuracy. With the NTU-RGBD dataset, we conducted the experiments on three advanced GCN-based models as baseline models. Analysis of the experiment results has proven that by adding the proposed network layer, the classification accuracy of human actions of all three baseline models improved. The enhanced performance indicates the effectiveness of frequency information in the task of skeleton action recognition, as well as the potential of attention mechanism in utilizing the frequency information. |
| first_indexed | 2024-10-01T04:11:27Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/172751 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:11:27Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1727512023-12-22T15:42:38Z Graph convolution network based skeleton action recognition with DCT features Hei, Hao Alex Chichung Kot School of Electrical and Electronic Engineering Rapid-Rich Object Search (ROSE) Lab EACKOT@ntu.edu.sg Engineering::Electrical and electronic engineering Human Action Recognition (HAR), which aims to decipher human movements from video, has been an important research topic in computer vision for many years, as it serves as the foundation for many innovative technologies and applications. While most recent HAR-related research focused on applying Graph Convolutional Networks (GCNs) on skeleton modality, little attention has been paid to taking advantage of the frequency representation of skeleton data. In this project, our objective is to study the effect of utilizing skeleton features in the frequency domain to perform HAR with GCN. To achieve the target, we first conduct a thorough review of current approaches for HAR and frequency analysis. Inspired by research on attention mechanism, we proposed to combine channel attention and 2-D Discrete Cosine Transform (DCT) as a universal layer of a deep learning network to utilize the frequency information from skeleton data, which can be inserted in the current GCNs for improvements in classification accuracy. With the NTU-RGBD dataset, we conducted the experiments on three advanced GCN-based models as baseline models. Analysis of the experiment results has proven that by adding the proposed network layer, the classification accuracy of human actions of all three baseline models improved. The enhanced performance indicates the effectiveness of frequency information in the task of skeleton action recognition, as well as the potential of attention mechanism in utilizing the frequency information. Bachelor of Engineering (Electrical and Electronic Engineering) 2023-12-19T07:59:15Z 2023-12-19T07:59:15Z 2023 Final Year Project (FYP) Hei, H. (2023). Graph convolution network based skeleton action recognition with DCT features. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/172751 https://hdl.handle.net/10356/172751 en A3297-222 application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering Hei, Hao Graph convolution network based skeleton action recognition with DCT features |
| title | Graph convolution network based skeleton action recognition with DCT features |
| title_full | Graph convolution network based skeleton action recognition with DCT features |
| title_fullStr | Graph convolution network based skeleton action recognition with DCT features |
| title_full_unstemmed | Graph convolution network based skeleton action recognition with DCT features |
| title_short | Graph convolution network based skeleton action recognition with DCT features |
| title_sort | graph convolution network based skeleton action recognition with dct features |
| topic | Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/172751 |
| work_keys_str_mv | AT heihao graphconvolutionnetworkbasedskeletonactionrecognitionwithdctfeatures |