Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition
Graph Convolutional Networks (GCNs) have attracted a lot of attention and shown remarkable performance for action recognition in recent years. For improving the recognition accuracy, how to build graph structure adaptively, select key frames and extract discriminative features are the key problems o...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/20/6761 |
| _version_ | 1797513178708443136 |
|---|---|
| author | Di Liu Hui Xu Jianzhong Wang Yinghua Lu Jun Kong Miao Qi |
| author_facet | Di Liu Hui Xu Jianzhong Wang Yinghua Lu Jun Kong Miao Qi |
| author_sort | Di Liu |
| collection | DOAJ |
| description | Graph Convolutional Networks (GCNs) have attracted a lot of attention and shown remarkable performance for action recognition in recent years. For improving the recognition accuracy, how to build graph structure adaptively, select key frames and extract discriminative features are the key problems of this kind of method. In this work, we propose a novel Adaptive Attention Memory Graph Convolutional Networks (AAM-GCN) for human action recognition using skeleton data. We adopt GCN to adaptively model the spatial configuration of skeletons and employ Gated Recurrent Unit (GRU) to construct an attention-enhanced memory for capturing the temporal feature. With the memory module, our model can not only remember what happened in the past but also employ the information in the future using multi-bidirectional GRU layers. Furthermore, in order to extract discriminative temporal features, the attention mechanism is also employed to select key frames from the skeleton sequence. Extensive experiments on Kinetics, NTU RGB+D and HDM05 datasets show that the proposed network achieves better performance than some state-of-the-art methods. |
| first_indexed | 2024-03-10T06:13:01Z |
| format | Article |
| id | doaj.art-160a420ae0484d07aac995e9facb0424 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T06:13:01Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-160a420ae0484d07aac995e9facb04242023-11-22T19:57:01ZengMDPI AGSensors1424-82202021-10-012120676110.3390/s21206761Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action RecognitionDi Liu0Hui Xu1Jianzhong Wang2Yinghua Lu3Jun Kong4Miao Qi5College of Information Sciences and Technology, Northeast Normal University, Changchun 130117, ChinaCollege of Information Sciences and Technology, Northeast Normal University, Changchun 130117, ChinaCollege of Information Sciences and Technology, Northeast Normal University, Changchun 130117, ChinaCollege of Information Sciences and Technology, Northeast Normal University, Changchun 130117, ChinaInstitute for Intelligent Elderly Care, Changchun Humanities and Sciences College, Changchun 130117, ChinaCollege of Information Sciences and Technology, Northeast Normal University, Changchun 130117, ChinaGraph Convolutional Networks (GCNs) have attracted a lot of attention and shown remarkable performance for action recognition in recent years. For improving the recognition accuracy, how to build graph structure adaptively, select key frames and extract discriminative features are the key problems of this kind of method. In this work, we propose a novel Adaptive Attention Memory Graph Convolutional Networks (AAM-GCN) for human action recognition using skeleton data. We adopt GCN to adaptively model the spatial configuration of skeletons and employ Gated Recurrent Unit (GRU) to construct an attention-enhanced memory for capturing the temporal feature. With the memory module, our model can not only remember what happened in the past but also employ the information in the future using multi-bidirectional GRU layers. Furthermore, in order to extract discriminative temporal features, the attention mechanism is also employed to select key frames from the skeleton sequence. Extensive experiments on Kinetics, NTU RGB+D and HDM05 datasets show that the proposed network achieves better performance than some state-of-the-art methods.https://www.mdpi.com/1424-8220/21/20/6761graph convolutional networksaction recognitionattentionskeleton sequence |
| spellingShingle | Di Liu Hui Xu Jianzhong Wang Yinghua Lu Jun Kong Miao Qi Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition Sensors graph convolutional networks action recognition attention skeleton sequence |
| title | Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_full | Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_fullStr | Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_full_unstemmed | Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_short | Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition |
| title_sort | adaptive attention memory graph convolutional networks for skeleton based action recognition |
| topic | graph convolutional networks action recognition attention skeleton sequence |
| url | https://www.mdpi.com/1424-8220/21/20/6761 |
| work_keys_str_mv | AT diliu adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition AT huixu adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition AT jianzhongwang adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition AT yinghualu adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition AT junkong adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition AT miaoqi adaptiveattentionmemorygraphconvolutionalnetworksforskeletonbasedactionrecognition |