A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection
In this paper, we propose a new method for detecting abnormal human behavior based on skeleton features using self-attention augment graph convolution. The skeleton data have been proved to be robust to the complex background, illumination changes, and dynamic camera scenes and are naturally constru...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/1/4 |
| _version_ | 1797499705472581632 |
|---|---|
| author | Chengming Liu Ronghua Fu Yinghao Li Yufei Gao Lei Shi Weiwei Li |
| author_facet | Chengming Liu Ronghua Fu Yinghao Li Yufei Gao Lei Shi Weiwei Li |
| author_sort | Chengming Liu |
| collection | DOAJ |
| description | In this paper, we propose a new method for detecting abnormal human behavior based on skeleton features using self-attention augment graph convolution. The skeleton data have been proved to be robust to the complex background, illumination changes, and dynamic camera scenes and are naturally constructed as a graph in non-Euclidean space. Particularly, the establishment of spatial temporal graph convolutional networks (ST-GCN) can effectively learn the spatio-temporal relationships of Non-Euclidean Structure Data. However, it only operates on local neighborhood nodes and thereby lacks global information. We propose a novel spatial temporal self-attention augmented graph convolutional networks (SAA-Graph) by combining improved spatial graph convolution operator with a modified transformer self-attention operator to capture both local and global information of the joints. The spatial self-attention augmented module is used to understand the intra-frame relationships between human body parts. As far as we know, we are the first group to utilize self-attention for video anomaly detection tasks by enhancing spatial temporal graph convolution. Moreover, to validate the proposed model, we performed extensive experiments on two large-scale publicly standard datasets (i.e., ShanghaiTech Campus and CUHK Avenue datasets) which reveal the state-of-art performance for our proposed approach when compared to existing skeleton-based methods and graph convolution methods. |
| first_indexed | 2024-03-10T03:51:16Z |
| format | Article |
| id | doaj.art-6a36eef2ecd4450a912de5558e1cd5a8 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T03:51:16Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6a36eef2ecd4450a912de5558e1cd5a82023-11-23T11:06:10ZengMDPI AGApplied Sciences2076-34172021-12-01121410.3390/app12010004A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior DetectionChengming Liu0Ronghua Fu1Yinghao Li2Yufei Gao3Lei Shi4Weiwei Li5School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, ChinaSchool of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, ChinaSchool of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, ChinaSchool of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, ChinaSchool of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, ChinaPrincipal’s Office, Zhengzhou College of Finance and Economics, Zhengzhou 450044, ChinaIn this paper, we propose a new method for detecting abnormal human behavior based on skeleton features using self-attention augment graph convolution. The skeleton data have been proved to be robust to the complex background, illumination changes, and dynamic camera scenes and are naturally constructed as a graph in non-Euclidean space. Particularly, the establishment of spatial temporal graph convolutional networks (ST-GCN) can effectively learn the spatio-temporal relationships of Non-Euclidean Structure Data. However, it only operates on local neighborhood nodes and thereby lacks global information. We propose a novel spatial temporal self-attention augmented graph convolutional networks (SAA-Graph) by combining improved spatial graph convolution operator with a modified transformer self-attention operator to capture both local and global information of the joints. The spatial self-attention augmented module is used to understand the intra-frame relationships between human body parts. As far as we know, we are the first group to utilize self-attention for video anomaly detection tasks by enhancing spatial temporal graph convolution. Moreover, to validate the proposed model, we performed extensive experiments on two large-scale publicly standard datasets (i.e., ShanghaiTech Campus and CUHK Avenue datasets) which reveal the state-of-art performance for our proposed approach when compared to existing skeleton-based methods and graph convolution methods.https://www.mdpi.com/2076-3417/12/1/4video anomaly detectionsskeletonself-attentiongraph convolutional networks |
| spellingShingle | Chengming Liu Ronghua Fu Yinghao Li Yufei Gao Lei Shi Weiwei Li A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection Applied Sciences video anomaly detections skeleton self-attention graph convolutional networks |
| title | A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection |
| title_full | A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection |
| title_fullStr | A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection |
| title_full_unstemmed | A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection |
| title_short | A Self-Attention Augmented Graph Convolutional Clustering Networks for Skeleton-Based Video Anomaly Behavior Detection |
| title_sort | self attention augmented graph convolutional clustering networks for skeleton based video anomaly behavior detection |
| topic | video anomaly detections skeleton self-attention graph convolutional networks |
| url | https://www.mdpi.com/2076-3417/12/1/4 |
| work_keys_str_mv | AT chengmingliu aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT ronghuafu aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT yinghaoli aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT yufeigao aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT leishi aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT weiweili aselfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT chengmingliu selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT ronghuafu selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT yinghaoli selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT yufeigao selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT leishi selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection AT weiweili selfattentionaugmentedgraphconvolutionalclusteringnetworksforskeletonbasedvideoanomalybehaviordetection |