Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints
A fine-grained detection of posture problems for action assessment has a wide range of applications for health care, sports, and rehabilitation. However, there exist many design challenges, e.g., the difficulty of detecting subtle deviations in actions from standard ones, lack of annotated datasets,...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10217805/ |
| _version_ | 1797736970643832832 |
|---|---|
| author | Chung-In Joung Seunghwan Byun Seungjun Baek |
| author_facet | Chung-In Joung Seunghwan Byun Seungjun Baek |
| author_sort | Chung-In Joung |
| collection | DOAJ |
| description | A fine-grained detection of posture problems for action assessment has a wide range of applications for health care, sports, and rehabilitation. However, there exist many design challenges, e.g., the difficulty of detecting subtle deviations in actions from standard ones, lack of annotated datasets, and even multiple posture problems that may be present in a single action. In this paper, we propose a contrastive learning framework leveraging graph convolutional networks to address these challenges. We introduce Augmented Virtual Joint which is a learned position in space where its associated graphs provide a holistic view of spatio-temporal dynamics of body joints, offering a flexible and generalized representation of actions. Next, we propose Degraded Negative Contrasting, which judiciously contrasts incorrect action samples for effective discrimination of incorrect actions from correct ones. We also propose Frame-Selective Pooling which provides a simple yet effective selection of important frames from action clips. Experiments show that, as compared with the state-of-the-art architectures, the proposed model consistently achieves the best performance under a lack of training data and in the presence of multiple posture problems, which demonstrates its efficacy for fine-grained evaluation of actions. |
| first_indexed | 2024-03-12T13:21:37Z |
| format | Article |
| id | doaj.art-aa9d10dfa0274b10b07c1014f0f8ef79 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T13:21:37Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-aa9d10dfa0274b10b07c1014f0f8ef792023-08-25T23:01:08ZengIEEEIEEE Access2169-35362023-01-0111888958890710.1109/ACCESS.2023.330537210217805Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual JointsChung-In Joung0https://orcid.org/0000-0002-5389-0508Seunghwan Byun1Seungjun Baek2https://orcid.org/0000-0002-1226-0147Korea University, Seongbuk-gu, South KoreaKorea University, Seongbuk-gu, South KoreaKorea University, Seongbuk-gu, South KoreaA fine-grained detection of posture problems for action assessment has a wide range of applications for health care, sports, and rehabilitation. However, there exist many design challenges, e.g., the difficulty of detecting subtle deviations in actions from standard ones, lack of annotated datasets, and even multiple posture problems that may be present in a single action. In this paper, we propose a contrastive learning framework leveraging graph convolutional networks to address these challenges. We introduce Augmented Virtual Joint which is a learned position in space where its associated graphs provide a holistic view of spatio-temporal dynamics of body joints, offering a flexible and generalized representation of actions. Next, we propose Degraded Negative Contrasting, which judiciously contrasts incorrect action samples for effective discrimination of incorrect actions from correct ones. We also propose Frame-Selective Pooling which provides a simple yet effective selection of important frames from action clips. Experiments show that, as compared with the state-of-the-art architectures, the proposed model consistently achieves the best performance under a lack of training data and in the presence of multiple posture problems, which demonstrates its efficacy for fine-grained evaluation of actions.https://ieeexplore.ieee.org/document/10217805/Graph convolutional networksaugmented virtual jointscontrastive learningpoolingfine-grained action classification |
| spellingShingle | Chung-In Joung Seunghwan Byun Seungjun Baek Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints IEEE Access Graph convolutional networks augmented virtual joints contrastive learning pooling fine-grained action classification |
| title | Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints |
| title_full | Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints |
| title_fullStr | Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints |
| title_full_unstemmed | Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints |
| title_short | Contrastive Learning for Action Assessment Using Graph Convolutional Networks With Augmented Virtual Joints |
| title_sort | contrastive learning for action assessment using graph convolutional networks with augmented virtual joints |
| topic | Graph convolutional networks augmented virtual joints contrastive learning pooling fine-grained action classification |
| url | https://ieeexplore.ieee.org/document/10217805/ |
| work_keys_str_mv | AT chunginjoung contrastivelearningforactionassessmentusinggraphconvolutionalnetworkswithaugmentedvirtualjoints AT seunghwanbyun contrastivelearningforactionassessmentusinggraphconvolutionalnetworkswithaugmentedvirtualjoints AT seungjunbaek contrastivelearningforactionassessmentusinggraphconvolutionalnetworkswithaugmentedvirtualjoints |