The Sub-Sequence Summary Method for Detecting Anomalies in Logs
This paper introduces a novel method for detecting log anomalies using deep learning. In contrast to state-of-the-art methods that rely on sequence models such as LSTMs or Transformers, our approach does not require the subsequent log lines to be fed directly into the model. Instead, we extract spec...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10102551/ |
| _version_ | 1797839162871644160 |
|---|---|
| author | Gabor Horvath Attila Kadar Peter Szilagyi |
| author_facet | Gabor Horvath Attila Kadar Peter Szilagyi |
| author_sort | Gabor Horvath |
| collection | DOAJ |
| description | This paper introduces a novel method for detecting log anomalies using deep learning. In contrast to state-of-the-art methods that rely on sequence models such as LSTMs or Transformers, our approach does not require the subsequent log lines to be fed directly into the model. Instead, we extract specific features from the log sequence, and derive anomaly scores from the reconstruction loss of an ordinary auto-encoder. These features are easy to obtain and contain sequential information. The presented method can detect both sequence and attribute anomalies using a single integrated model. We present two variants: a template-based method and a fully semantic-based method. |
| first_indexed | 2024-04-09T15:53:42Z |
| format | Article |
| id | doaj.art-1c8f527697ce4bbdace7ab76d5c872da |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T15:53:42Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1c8f527697ce4bbdace7ab76d5c872da2023-04-25T23:00:55ZengIEEEIEEE Access2169-35362023-01-0111374123742310.1109/ACCESS.2023.326699010102551The Sub-Sequence Summary Method for Detecting Anomalies in LogsGabor Horvath0https://orcid.org/0000-0003-3097-1273Attila Kadar1https://orcid.org/0009-0007-8798-950XPeter Szilagyi2https://orcid.org/0000-0003-2106-6343Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, HungaryDepartment of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, HungaryNokia Bell Labs, Budapest, HungaryThis paper introduces a novel method for detecting log anomalies using deep learning. In contrast to state-of-the-art methods that rely on sequence models such as LSTMs or Transformers, our approach does not require the subsequent log lines to be fed directly into the model. Instead, we extract specific features from the log sequence, and derive anomaly scores from the reconstruction loss of an ordinary auto-encoder. These features are easy to obtain and contain sequential information. The presented method can detect both sequence and attribute anomalies using a single integrated model. We present two variants: a template-based method and a fully semantic-based method.https://ieeexplore.ieee.org/document/10102551/Log analyticsanomaly detectionsequence anomalyattribute anomalyauto-encoder |
| spellingShingle | Gabor Horvath Attila Kadar Peter Szilagyi The Sub-Sequence Summary Method for Detecting Anomalies in Logs IEEE Access Log analytics anomaly detection sequence anomaly attribute anomaly auto-encoder |
| title | The Sub-Sequence Summary Method for Detecting Anomalies in Logs |
| title_full | The Sub-Sequence Summary Method for Detecting Anomalies in Logs |
| title_fullStr | The Sub-Sequence Summary Method for Detecting Anomalies in Logs |
| title_full_unstemmed | The Sub-Sequence Summary Method for Detecting Anomalies in Logs |
| title_short | The Sub-Sequence Summary Method for Detecting Anomalies in Logs |
| title_sort | sub sequence summary method for detecting anomalies in logs |
| topic | Log analytics anomaly detection sequence anomaly attribute anomaly auto-encoder |
| url | https://ieeexplore.ieee.org/document/10102551/ |
| work_keys_str_mv | AT gaborhorvath thesubsequencesummarymethodfordetectinganomaliesinlogs AT attilakadar thesubsequencesummarymethodfordetectinganomaliesinlogs AT peterszilagyi thesubsequencesummarymethodfordetectinganomaliesinlogs AT gaborhorvath subsequencesummarymethodfordetectinganomaliesinlogs AT attilakadar subsequencesummarymethodfordetectinganomaliesinlogs AT peterszilagyi subsequencesummarymethodfordetectinganomaliesinlogs |