Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition
Previous research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/15/3280 |
| _version_ | 1797586351452848128 |
|---|---|
| author | Jinyoon Park Chulwoong Kim Seung-Chan Kim |
| author_facet | Jinyoon Park Chulwoong Kim Seung-Chan Kim |
| author_sort | Jinyoon Park |
| collection | DOAJ |
| description | Previous research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found in short-term videos, a characteristic commonly encountered in public datasets. However, our preliminary investigation of complex action sequences, such as discussions or smoking, reveals its limitations in capturing the intricacies of such actions. To address these view-dependency issues, we propose a straightforward, yet effective, sequence-wise augmentation technique. This strategy enhances the robustness of action recognition models, particularly against changes in viewing direction that mainly occur within the horizontal plane (azimuth) by rotating human key points around either the z-axis or the spine vector, effectively creating variations in viewing directions. We scrutinize the robustness of this approach against real-world viewpoint variations through extensive empirical studies on multiple public datasets, including an additional set of custom action sequences. Despite the simplicity of our approach, our experimental results consistently yield improved action recognition accuracies. Compared to the sequence-wise viewpoint normalization method used with advanced deep learning models like Conv1D, LSTM, and Transformer, our approach showed a relative increase in accuracy of 34.42% for the z-axis and 10.86% for the spine vector. |
| first_indexed | 2024-03-11T00:22:04Z |
| format | Article |
| id | doaj.art-05a978099c9545eeabcc65559fa25d07 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T00:22:04Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-05a978099c9545eeabcc65559fa25d072023-11-18T23:14:32ZengMDPI AGMathematics2227-73902023-07-011115328010.3390/math11153280Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action RecognitionJinyoon Park0Chulwoong Kim1Seung-Chan Kim2Machine Learning Systems Lab., Department of Sport Interaction Science, Sungkyunkwan University, Suwon 16419, Republic of KoreaTAIIPA—Taean AI Industry Promotion Agency, Taean 32154, Republic of KoreaMachine Learning Systems Lab., Department of Sport Interaction Science, Sungkyunkwan University, Suwon 16419, Republic of KoreaPrevious research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found in short-term videos, a characteristic commonly encountered in public datasets. However, our preliminary investigation of complex action sequences, such as discussions or smoking, reveals its limitations in capturing the intricacies of such actions. To address these view-dependency issues, we propose a straightforward, yet effective, sequence-wise augmentation technique. This strategy enhances the robustness of action recognition models, particularly against changes in viewing direction that mainly occur within the horizontal plane (azimuth) by rotating human key points around either the z-axis or the spine vector, effectively creating variations in viewing directions. We scrutinize the robustness of this approach against real-world viewpoint variations through extensive empirical studies on multiple public datasets, including an additional set of custom action sequences. Despite the simplicity of our approach, our experimental results consistently yield improved action recognition accuracies. Compared to the sequence-wise viewpoint normalization method used with advanced deep learning models like Conv1D, LSTM, and Transformer, our approach showed a relative increase in accuracy of 34.42% for the z-axis and 10.86% for the spine vector.https://www.mdpi.com/2227-7390/11/15/3280action recognitionmachine learningfeature learningskeletal datadata augmentation |
| spellingShingle | Jinyoon Park Chulwoong Kim Seung-Chan Kim Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition Mathematics action recognition machine learning feature learning skeletal data data augmentation |
| title | Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition |
| title_full | Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition |
| title_fullStr | Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition |
| title_full_unstemmed | Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition |
| title_short | Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition |
| title_sort | enhancing robustness of viewpoint changes in 3d skeleton based human action recognition |
| topic | action recognition machine learning feature learning skeletal data data augmentation |
| url | https://www.mdpi.com/2227-7390/11/15/3280 |
| work_keys_str_mv | AT jinyoonpark enhancingrobustnessofviewpointchangesin3dskeletonbasedhumanactionrecognition AT chulwoongkim enhancingrobustnessofviewpointchangesin3dskeletonbasedhumanactionrecognition AT seungchankim enhancingrobustnessofviewpointchangesin3dskeletonbasedhumanactionrecognition |