Large scale log anomaly detection via spatial pooling
Log data is an important clue to understanding the behaviour of a system at runtime, but the complexity of software systems in recent years has made the data that engineers need to analyse enormous and difficult to understand. While log-based anomaly detection methods based on deep learning have ena...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2021-01-01
|
| Series: | Cognitive Robotics |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667241321000173 |
| _version_ | 1797976411922759680 |
|---|---|
| author | Rin Hirakawa Hironori Uchida Asato Nakano Keitaro Tominaga Yoshihisa Nakatoh |
| author_facet | Rin Hirakawa Hironori Uchida Asato Nakano Keitaro Tominaga Yoshihisa Nakatoh |
| author_sort | Rin Hirakawa |
| collection | DOAJ |
| description | Log data is an important clue to understanding the behaviour of a system at runtime, but the complexity of software systems in recent years has made the data that engineers need to analyse enormous and difficult to understand. While log-based anomaly detection methods based on deep learning have enabled highly accurate detection, the computational performance required to operate the models has become very high. In this study, we propose an anomaly detection method, SPClassifier, based on sparse features and the internal state of the model, and investigate the feasibility of anomaly detection that can be utilized in environments without computing resources such as GPUs. Benchmark with the latest deep learning models on the BGL dataset shows that the proposed method can achieve competitive accuracy with these methods and has a high level of anomaly detection performance even when the amount of training data is small. |
| first_indexed | 2024-04-11T04:50:31Z |
| format | Article |
| id | doaj.art-e65e31fda24e4dbb97dfd7c77b9b02d8 |
| institution | Directory Open Access Journal |
| issn | 2667-2413 |
| language | English |
| last_indexed | 2024-04-11T04:50:31Z |
| publishDate | 2021-01-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | Cognitive Robotics |
| spelling | doaj.art-e65e31fda24e4dbb97dfd7c77b9b02d82022-12-27T04:41:33ZengKeAi Communications Co. Ltd.Cognitive Robotics2667-24132021-01-011188196Large scale log anomaly detection via spatial poolingRin Hirakawa0Hironori Uchida1Asato Nakano2Keitaro Tominaga3Yoshihisa Nakatoh4Kyushu Institute of Technology, Fukuoka, JapanPanasonic System Design Co., Ltd., Kanagawa, JapanPanasonic System Design Co., Ltd., Kanagawa, JapanPanasonic System Design Co., Ltd., Kanagawa, JapanKyushu Institute of Technology, Fukuoka, Japan; Corresponding author.Log data is an important clue to understanding the behaviour of a system at runtime, but the complexity of software systems in recent years has made the data that engineers need to analyse enormous and difficult to understand. While log-based anomaly detection methods based on deep learning have enabled highly accurate detection, the computational performance required to operate the models has become very high. In this study, we propose an anomaly detection method, SPClassifier, based on sparse features and the internal state of the model, and investigate the feasibility of anomaly detection that can be utilized in environments without computing resources such as GPUs. Benchmark with the latest deep learning models on the BGL dataset shows that the proposed method can achieve competitive accuracy with these methods and has a high level of anomaly detection performance even when the amount of training data is small.http://www.sciencedirect.com/science/article/pii/S2667241321000173Anomaly detectionSoftware logSpatial poolingDefect analysis |
| spellingShingle | Rin Hirakawa Hironori Uchida Asato Nakano Keitaro Tominaga Yoshihisa Nakatoh Large scale log anomaly detection via spatial pooling Cognitive Robotics Anomaly detection Software log Spatial pooling Defect analysis |
| title | Large scale log anomaly detection via spatial pooling |
| title_full | Large scale log anomaly detection via spatial pooling |
| title_fullStr | Large scale log anomaly detection via spatial pooling |
| title_full_unstemmed | Large scale log anomaly detection via spatial pooling |
| title_short | Large scale log anomaly detection via spatial pooling |
| title_sort | large scale log anomaly detection via spatial pooling |
| topic | Anomaly detection Software log Spatial pooling Defect analysis |
| url | http://www.sciencedirect.com/science/article/pii/S2667241321000173 |
| work_keys_str_mv | AT rinhirakawa largescaleloganomalydetectionviaspatialpooling AT hironoriuchida largescaleloganomalydetectionviaspatialpooling AT asatonakano largescaleloganomalydetectionviaspatialpooling AT keitarotominaga largescaleloganomalydetectionviaspatialpooling AT yoshihisanakatoh largescaleloganomalydetectionviaspatialpooling |