Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos
Anomaly detection in videos is the task of identifying frames from a video sequence that depict events that do not conform to expected behavior, which is an extremely challenging task due to the ambiguous and unbounded properties of anomalies. With the development of deep learning, video anomaly det...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9499069/ |
| _version_ | 1818607527283130368 |
|---|---|
| author | Zhiwei Yang Jing Liu Peng Wu |
| author_facet | Zhiwei Yang Jing Liu Peng Wu |
| author_sort | Zhiwei Yang |
| collection | DOAJ |
| description | Anomaly detection in videos is the task of identifying frames from a video sequence that depict events that do not conform to expected behavior, which is an extremely challenging task due to the ambiguous and unbounded properties of anomalies. With the development of deep learning, video anomaly detection methods based on deep neural networks have made great progress. The existing methods mainly follow two routes, namely, frame reconstruction and frame prediction. Due to the powerful generalization ability of neural networks, the application of reconstruction-based methods is limited. Recently, anomaly detection methods based on prediction have achieved advanced performance. However, their performance suffers when they cannot guarantee lower prediction errors for normal events. In this paper, we propose a novel future frame prediction model based on a bidirectional retrospective generation adversarial network (BR-GAN) for anomaly detection. To predict a future frame with higher quality for normal events, first, we propose a bidirectional prediction combined with a retrospective prediction method to fully mine the bidirectional temporal information between the predicted frame and the input frame sequence. Then, the intensity and gradient loss between the predicted frame and the actual frame together with an adversarial loss are used for appearance (spatial) constraints. In addition, we propose a sequence discriminator composed of a 3-dimensional (3D) convolutional neural network to capture the long-term temporal relationships between frame sequences composed of predicted frames and input frames; this network plays a crucial role in maintaining the motion (temporal) consistency of the predicted frames for normal events. Such appearance and motion constraints further facilitate future frame prediction for normal events, and thus, the prediction network can be highly capable of distinguishing normal and abnormal patterns. Extensive experiments on benchmark datasets demonstrate that our method outperforms most existing state-of-the-art methods, validating the effectiveness of our method for anomaly detection. |
| first_indexed | 2024-12-16T14:28:10Z |
| format | Article |
| id | doaj.art-1401b0711fe84b96b4bbd91a73a434d5 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T14:28:10Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1401b0711fe84b96b4bbd91a73a434d52022-12-21T22:28:19ZengIEEEIEEE Access2169-35362021-01-01910784210785710.1109/ACCESS.2021.31006789499069Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in VideosZhiwei Yang0https://orcid.org/0000-0001-9896-4518Jing Liu1https://orcid.org/0000-0002-6834-5350Peng Wu2https://orcid.org/0000-0003-2938-6798Guangzhou Institute of Technology, Xidian University, Guangzhou, ChinaGuangzhou Institute of Technology, Xidian University, Guangzhou, ChinaGuangzhou Institute of Technology, Xidian University, Guangzhou, ChinaAnomaly detection in videos is the task of identifying frames from a video sequence that depict events that do not conform to expected behavior, which is an extremely challenging task due to the ambiguous and unbounded properties of anomalies. With the development of deep learning, video anomaly detection methods based on deep neural networks have made great progress. The existing methods mainly follow two routes, namely, frame reconstruction and frame prediction. Due to the powerful generalization ability of neural networks, the application of reconstruction-based methods is limited. Recently, anomaly detection methods based on prediction have achieved advanced performance. However, their performance suffers when they cannot guarantee lower prediction errors for normal events. In this paper, we propose a novel future frame prediction model based on a bidirectional retrospective generation adversarial network (BR-GAN) for anomaly detection. To predict a future frame with higher quality for normal events, first, we propose a bidirectional prediction combined with a retrospective prediction method to fully mine the bidirectional temporal information between the predicted frame and the input frame sequence. Then, the intensity and gradient loss between the predicted frame and the actual frame together with an adversarial loss are used for appearance (spatial) constraints. In addition, we propose a sequence discriminator composed of a 3-dimensional (3D) convolutional neural network to capture the long-term temporal relationships between frame sequences composed of predicted frames and input frames; this network plays a crucial role in maintaining the motion (temporal) consistency of the predicted frames for normal events. Such appearance and motion constraints further facilitate future frame prediction for normal events, and thus, the prediction network can be highly capable of distinguishing normal and abnormal patterns. Extensive experiments on benchmark datasets demonstrate that our method outperforms most existing state-of-the-art methods, validating the effectiveness of our method for anomaly detection.https://ieeexplore.ieee.org/document/9499069/Anomaly detectionfuture frame predictiongeneration adversarial networkvideo surveillance |
| spellingShingle | Zhiwei Yang Jing Liu Peng Wu Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos IEEE Access Anomaly detection future frame prediction generation adversarial network video surveillance |
| title | Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos |
| title_full | Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos |
| title_fullStr | Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos |
| title_full_unstemmed | Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos |
| title_short | Bidirectional Retrospective Generation Adversarial Network for Anomaly Detection in Videos |
| title_sort | bidirectional retrospective generation adversarial network for anomaly detection in videos |
| topic | Anomaly detection future frame prediction generation adversarial network video surveillance |
| url | https://ieeexplore.ieee.org/document/9499069/ |
| work_keys_str_mv | AT zhiweiyang bidirectionalretrospectivegenerationadversarialnetworkforanomalydetectioninvideos AT jingliu bidirectionalretrospectivegenerationadversarialnetworkforanomalydetectioninvideos AT pengwu bidirectionalretrospectivegenerationadversarialnetworkforanomalydetectioninvideos |