Deep Reinforcement Learning for Anomaly Detection: A Systematic Review
Anomaly detection has been used to detect and analyze anomalous elements from data for years. Various techniques have been developed to detect anomalies. However, the most convenient one is Machine learning which is performing well but still has limitations for large-scale unlabeled datasets. Deep R...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9956995/ |
| _version_ | 1811205168797057024 |
|---|---|
| author | Kinza Arshad Rao Faizan Ali Amgad Muneer Izzatdin Abdul Aziz Sheraz Naseer Nabeel Sabir Khan Shakirah Mohd Taib |
| author_facet | Kinza Arshad Rao Faizan Ali Amgad Muneer Izzatdin Abdul Aziz Sheraz Naseer Nabeel Sabir Khan Shakirah Mohd Taib |
| author_sort | Kinza Arshad |
| collection | DOAJ |
| description | Anomaly detection has been used to detect and analyze anomalous elements from data for years. Various techniques have been developed to detect anomalies. However, the most convenient one is Machine learning which is performing well but still has limitations for large-scale unlabeled datasets. Deep Reinforcement Learning (DRL) based techniques outperform the existing supervised or unsupervised and other alternative techniques for anomaly detection. This study presents a Systematic Literature Review (SLR), which analyzes DRL models that detect anomalies in their application. This SLR aims to analyze the DRL frameworks for anomaly detection applications, proposed DRL methods, and their performance comparisons against alternative methods. In this review, we have identified 32 research articles published from 2017–2022 that discuss DRL techniques for various anomaly detection applications. After analyzing the selected research articles, this paper presents 13 different applications of anomaly detection found in the selected research articles. We identified 50 different datasets applied in experiments on anomaly detection and demonstrated 17 distinct DRL models used in the selected papers to detect anomalies. Finally, we analyzed the performance of these DRL models and reviewed them. Additionally, we observed that detecting anomalies using DRL frameworks is a promising area of research and showed that DRL had shown better performance for anomaly detection where other models lack. Therefore, we provide researchers with recommendations and guidelines based on this review. |
| first_indexed | 2024-04-12T03:26:10Z |
| format | Article |
| id | doaj.art-7138e268ceac4457b849fdf53e1acaa4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T03:26:10Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-7138e268ceac4457b849fdf53e1acaa42022-12-22T03:49:41ZengIEEEIEEE Access2169-35362022-01-011012401712403510.1109/ACCESS.2022.32240239956995Deep Reinforcement Learning for Anomaly Detection: A Systematic ReviewKinza Arshad0Rao Faizan Ali1https://orcid.org/0000-0003-0701-6761Amgad Muneer2https://orcid.org/0000-0002-7157-3020Izzatdin Abdul Aziz3https://orcid.org/0000-0003-2654-4463Sheraz Naseer4https://orcid.org/0000-0002-3224-9164Nabeel Sabir Khan5https://orcid.org/0000-0003-0758-6019Shakirah Mohd Taib6Department of Software Engineering, University of Management and Technology, Lahore, PakistanDepartment of Software Engineering, University of Management and Technology, Lahore, PakistanDepartment of Computer and Information Sciences, Centre for Research in Data Science (CERDAS), Universiti Teknologi PETRONAS, Seri Iskandar, MalaysiaDepartment of Computer and Information Sciences, Centre for Research in Data Science (CERDAS), Universiti Teknologi PETRONAS, Seri Iskandar, MalaysiaDepartment of Computer Science, University of Engineering and Technology, KSK Campus, Lahore, PakistanDepartment of Computer Science, University of Central Punjab, Lahore, PakistanDepartment of Computer and Information Sciences, Centre for Research in Data Science (CERDAS), Universiti Teknologi PETRONAS, Seri Iskandar, MalaysiaAnomaly detection has been used to detect and analyze anomalous elements from data for years. Various techniques have been developed to detect anomalies. However, the most convenient one is Machine learning which is performing well but still has limitations for large-scale unlabeled datasets. Deep Reinforcement Learning (DRL) based techniques outperform the existing supervised or unsupervised and other alternative techniques for anomaly detection. This study presents a Systematic Literature Review (SLR), which analyzes DRL models that detect anomalies in their application. This SLR aims to analyze the DRL frameworks for anomaly detection applications, proposed DRL methods, and their performance comparisons against alternative methods. In this review, we have identified 32 research articles published from 2017–2022 that discuss DRL techniques for various anomaly detection applications. After analyzing the selected research articles, this paper presents 13 different applications of anomaly detection found in the selected research articles. We identified 50 different datasets applied in experiments on anomaly detection and demonstrated 17 distinct DRL models used in the selected papers to detect anomalies. Finally, we analyzed the performance of these DRL models and reviewed them. Additionally, we observed that detecting anomalies using DRL frameworks is a promising area of research and showed that DRL had shown better performance for anomaly detection where other models lack. Therefore, we provide researchers with recommendations and guidelines based on this review.https://ieeexplore.ieee.org/document/9956995/Anomaly detectiondeep reinforcement learningsystematic review |
| spellingShingle | Kinza Arshad Rao Faizan Ali Amgad Muneer Izzatdin Abdul Aziz Sheraz Naseer Nabeel Sabir Khan Shakirah Mohd Taib Deep Reinforcement Learning for Anomaly Detection: A Systematic Review IEEE Access Anomaly detection deep reinforcement learning systematic review |
| title | Deep Reinforcement Learning for Anomaly Detection: A Systematic Review |
| title_full | Deep Reinforcement Learning for Anomaly Detection: A Systematic Review |
| title_fullStr | Deep Reinforcement Learning for Anomaly Detection: A Systematic Review |
| title_full_unstemmed | Deep Reinforcement Learning for Anomaly Detection: A Systematic Review |
| title_short | Deep Reinforcement Learning for Anomaly Detection: A Systematic Review |
| title_sort | deep reinforcement learning for anomaly detection a systematic review |
| topic | Anomaly detection deep reinforcement learning systematic review |
| url | https://ieeexplore.ieee.org/document/9956995/ |
| work_keys_str_mv | AT kinzaarshad deepreinforcementlearningforanomalydetectionasystematicreview AT raofaizanali deepreinforcementlearningforanomalydetectionasystematicreview AT amgadmuneer deepreinforcementlearningforanomalydetectionasystematicreview AT izzatdinabdulaziz deepreinforcementlearningforanomalydetectionasystematicreview AT sheraznaseer deepreinforcementlearningforanomalydetectionasystematicreview AT nabeelsabirkhan deepreinforcementlearningforanomalydetectionasystematicreview AT shakirahmohdtaib deepreinforcementlearningforanomalydetectionasystematicreview |