Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism
Abstract Frame prediction methods based on Auto‐Encoder (AE) composed of convolutional neural networks (CNN) are very popular in detecting abnormal behaviour. The methods predict normal behaviour accurately and abnormal behaviour incorrectly, which is considered a criterion for abnormality discrimin...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-02-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12666 |
| _version_ | 1797903374783348736 |
|---|---|
| author | Xiaopeng Wen Huicheng Lai Guxue Gao Yanjie Zhao |
| author_facet | Xiaopeng Wen Huicheng Lai Guxue Gao Yanjie Zhao |
| author_sort | Xiaopeng Wen |
| collection | DOAJ |
| description | Abstract Frame prediction methods based on Auto‐Encoder (AE) composed of convolutional neural networks (CNN) are very popular in detecting abnormal behaviour. The methods predict normal behaviour accurately and abnormal behaviour incorrectly, which is considered a criterion for abnormality discrimination. However, the emergence of problems such as too strong AE representation leading to detection failure, the insufficient ability of the network to extract spatio‐temporal information, a large number of model parameters and slow running speed leads to the need for the method to be further improved. In this work, the authors propose a network framework for abnormal behaviour detection in video based on a pseudo‐3D encoder and a multi‐cascade memory mechanism (MMP3D). First of all, the encoder consisting of pseudo‐3D convolution is used to extract spatio‐temporal information from the video. Then, the multi‐cascade memory mechanism (MM) and the multi‐headed prototype attention mechanism are used to store and aggregate features of normal behaviour, which solves to some extent the problem of detection failure caused by strong AE representation power. Finally, the decoder designed by the 2D deconvolution layers is used to recover the prediction information. The efficiency and superiority of our method is validated on the Ped2 dataset, Avenue dataset, and ShanghaiTech dataset. |
| first_indexed | 2024-04-10T09:31:53Z |
| format | Article |
| id | doaj.art-50a7116a2812433c9bd7a997643245ea |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-04-10T09:31:53Z |
| publishDate | 2023-02-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-50a7116a2812433c9bd7a997643245ea2023-02-19T04:18:32ZengWileyIET Image Processing1751-96591751-96672023-02-0117370972110.1049/ipr2.12666Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanismXiaopeng Wen0Huicheng Lai1Guxue Gao2Yanjie Zhao3College of Information Science and Engineering Xinjiang University Urumqi ChinaCollege of Information Science and Engineering Xinjiang University Urumqi ChinaCollege of Information Science and Engineering Xinjiang University Urumqi ChinaCollege of Information Science and Engineering Xinjiang University Urumqi ChinaAbstract Frame prediction methods based on Auto‐Encoder (AE) composed of convolutional neural networks (CNN) are very popular in detecting abnormal behaviour. The methods predict normal behaviour accurately and abnormal behaviour incorrectly, which is considered a criterion for abnormality discrimination. However, the emergence of problems such as too strong AE representation leading to detection failure, the insufficient ability of the network to extract spatio‐temporal information, a large number of model parameters and slow running speed leads to the need for the method to be further improved. In this work, the authors propose a network framework for abnormal behaviour detection in video based on a pseudo‐3D encoder and a multi‐cascade memory mechanism (MMP3D). First of all, the encoder consisting of pseudo‐3D convolution is used to extract spatio‐temporal information from the video. Then, the multi‐cascade memory mechanism (MM) and the multi‐headed prototype attention mechanism are used to store and aggregate features of normal behaviour, which solves to some extent the problem of detection failure caused by strong AE representation power. Finally, the decoder designed by the 2D deconvolution layers is used to recover the prediction information. The efficiency and superiority of our method is validated on the Ped2 dataset, Avenue dataset, and ShanghaiTech dataset.https://doi.org/10.1049/ipr2.12666memory modulepseudo‐3D convolutionvideo abnormal behaviour detection |
| spellingShingle | Xiaopeng Wen Huicheng Lai Guxue Gao Yanjie Zhao Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism IET Image Processing memory module pseudo‐3D convolution video abnormal behaviour detection |
| title | Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism |
| title_full | Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism |
| title_fullStr | Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism |
| title_full_unstemmed | Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism |
| title_short | Video abnormal behaviour detection based on pseudo‐3D encoder and multi‐cascade memory mechanism |
| title_sort | video abnormal behaviour detection based on pseudo 3d encoder and multi cascade memory mechanism |
| topic | memory module pseudo‐3D convolution video abnormal behaviour detection |
| url | https://doi.org/10.1049/ipr2.12666 |
| work_keys_str_mv | AT xiaopengwen videoabnormalbehaviourdetectionbasedonpseudo3dencoderandmulticascadememorymechanism AT huichenglai videoabnormalbehaviourdetectionbasedonpseudo3dencoderandmulticascadememorymechanism AT guxuegao videoabnormalbehaviourdetectionbasedonpseudo3dencoderandmulticascadememorymechanism AT yanjiezhao videoabnormalbehaviourdetectionbasedonpseudo3dencoderandmulticascadememorymechanism |