Light-Weight Classification of Human Actions in Video with Skeleton-Based Features
An approach to human action classification in videos is presented, based on knowledge-aware initial features extracted from human skeleton data and on further processing by convolutional networks. The proposed smart tracking of skeleton joints, approximation of missing joints and normalization of sk...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/11/14/2145 |
| _version_ | 1797433701800345600 |
|---|---|
| author | Włodzimierz Kasprzak Bartłomiej Jankowski |
| author_facet | Włodzimierz Kasprzak Bartłomiej Jankowski |
| author_sort | Włodzimierz Kasprzak |
| collection | DOAJ |
| description | An approach to human action classification in videos is presented, based on knowledge-aware initial features extracted from human skeleton data and on further processing by convolutional networks. The proposed smart tracking of skeleton joints, approximation of missing joints and normalization of skeleton data are important steps of feature extraction. Three neural network models—based on LSTM, Transformer and CNN—are developed and experimentally verified. The models are trained and tested on the well-known NTU-RGB+D (Shahroudy et al., 2016) dataset in the cross-view mode. The obtained results show a competitive performance with other SOTA methods and verify the efficiency of proposed feature engineering. The network has a five times lower number of trainable parameters than other proposed methods to reach nearly similar performance and twenty times lower number than the currently best performing solutions. Thanks to the lightness of the classifier, the solution only requires relatively small computational resources. |
| first_indexed | 2024-03-09T10:20:45Z |
| format | Article |
| id | doaj.art-df48ae6483424f92a6886df0bd4814eb |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T10:20:45Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-df48ae6483424f92a6886df0bd4814eb2023-12-01T22:05:11ZengMDPI AGElectronics2079-92922022-07-011114214510.3390/electronics11142145Light-Weight Classification of Human Actions in Video with Skeleton-Based FeaturesWłodzimierz Kasprzak0Bartłomiej Jankowski1Institute of Control and Computation Engineering, Warsaw University of Technology, 00-661 Warszawa, PolandInstitute of Control and Computation Engineering, Warsaw University of Technology, 00-661 Warszawa, PolandAn approach to human action classification in videos is presented, based on knowledge-aware initial features extracted from human skeleton data and on further processing by convolutional networks. The proposed smart tracking of skeleton joints, approximation of missing joints and normalization of skeleton data are important steps of feature extraction. Three neural network models—based on LSTM, Transformer and CNN—are developed and experimentally verified. The models are trained and tested on the well-known NTU-RGB+D (Shahroudy et al., 2016) dataset in the cross-view mode. The obtained results show a competitive performance with other SOTA methods and verify the efficiency of proposed feature engineering. The network has a five times lower number of trainable parameters than other proposed methods to reach nearly similar performance and twenty times lower number than the currently best performing solutions. Thanks to the lightness of the classifier, the solution only requires relatively small computational resources.https://www.mdpi.com/2079-9292/11/14/2145human actionsneural network modelsskeleton datamachine learningNTU-RGBD datasetOpenPose |
| spellingShingle | Włodzimierz Kasprzak Bartłomiej Jankowski Light-Weight Classification of Human Actions in Video with Skeleton-Based Features Electronics human actions neural network models skeleton data machine learning NTU-RGBD dataset OpenPose |
| title | Light-Weight Classification of Human Actions in Video with Skeleton-Based Features |
| title_full | Light-Weight Classification of Human Actions in Video with Skeleton-Based Features |
| title_fullStr | Light-Weight Classification of Human Actions in Video with Skeleton-Based Features |
| title_full_unstemmed | Light-Weight Classification of Human Actions in Video with Skeleton-Based Features |
| title_short | Light-Weight Classification of Human Actions in Video with Skeleton-Based Features |
| title_sort | light weight classification of human actions in video with skeleton based features |
| topic | human actions neural network models skeleton data machine learning NTU-RGBD dataset OpenPose |
| url | https://www.mdpi.com/2079-9292/11/14/2145 |
| work_keys_str_mv | AT włodzimierzkasprzak lightweightclassificationofhumanactionsinvideowithskeletonbasedfeatures AT bartłomiejjankowski lightweightclassificationofhumanactionsinvideowithskeletonbasedfeatures |