Detecting Anomalies in System Logs With a Compact Convolutional Transformer
Computer systems play an important role to ensure the correct functioning of critical systems such as train stations, power stations, emergency systems, and server infrastructures. To ensure the correct functioning and safety of these computer systems, the detection of abnormal system behavior is cr...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10285328/ |
| _version_ | 1827789279723520000 |
|---|---|
| author | Rene Larisch Julien Vitay Fred H. Hamker |
| author_facet | Rene Larisch Julien Vitay Fred H. Hamker |
| author_sort | Rene Larisch |
| collection | DOAJ |
| description | Computer systems play an important role to ensure the correct functioning of critical systems such as train stations, power stations, emergency systems, and server infrastructures. To ensure the correct functioning and safety of these computer systems, the detection of abnormal system behavior is crucial. For that purpose, monitoring log data (mirroring the recent and current system status) is very commonly used. Because log data consists mainly of words and numbers, recent work used Transformer-based networks to analyze the log data and predict anomalies. Despite their success in fields such as natural language processing and computer vision, the main disadvantage of Transformers is the huge amount of trainable parameters, leading to long training times. In this work, we use a Compact Convolutional Transformer to detect anomalies in log data. Using convolutional layers leads to a much smaller number of trainable parameters and enable the processing of many consecutive log lines. We evaluate the proposed network on two standard datasets for log data anomaly detection, Blue Gene/L (BGL) and Spirit. Our results demonstrate that the combination of convolutional processing and self-attention improves the performance for anomaly detection in comparison to other self-supervised Transformer-based approaches, and is even on par with supervised approaches. |
| first_indexed | 2024-03-11T17:18:05Z |
| format | Article |
| id | doaj.art-ecd2a890aaed4831852ecca75aac72d0 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T17:18:05Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ecd2a890aaed4831852ecca75aac72d02023-10-19T23:00:42ZengIEEEIEEE Access2169-35362023-01-011111346411347910.1109/ACCESS.2023.332325210285328Detecting Anomalies in System Logs With a Compact Convolutional TransformerRene Larisch0https://orcid.org/0000-0003-3544-0631Julien Vitay1https://orcid.org/0000-0001-5229-2349Fred H. Hamker2https://orcid.org/0000-0001-9104-7143Department of Computer Science, Chemnitz University of Technology, Chemnitz, GermanyDepartment of Computer Science, Chemnitz University of Technology, Chemnitz, GermanyDepartment of Computer Science, Chemnitz University of Technology, Chemnitz, GermanyComputer systems play an important role to ensure the correct functioning of critical systems such as train stations, power stations, emergency systems, and server infrastructures. To ensure the correct functioning and safety of these computer systems, the detection of abnormal system behavior is crucial. For that purpose, monitoring log data (mirroring the recent and current system status) is very commonly used. Because log data consists mainly of words and numbers, recent work used Transformer-based networks to analyze the log data and predict anomalies. Despite their success in fields such as natural language processing and computer vision, the main disadvantage of Transformers is the huge amount of trainable parameters, leading to long training times. In this work, we use a Compact Convolutional Transformer to detect anomalies in log data. Using convolutional layers leads to a much smaller number of trainable parameters and enable the processing of many consecutive log lines. We evaluate the proposed network on two standard datasets for log data anomaly detection, Blue Gene/L (BGL) and Spirit. Our results demonstrate that the combination of convolutional processing and self-attention improves the performance for anomaly detection in comparison to other self-supervised Transformer-based approaches, and is even on par with supervised approaches.https://ieeexplore.ieee.org/document/10285328/Anomaly detectiondeep learningself-supervised learningtransformer |
| spellingShingle | Rene Larisch Julien Vitay Fred H. Hamker Detecting Anomalies in System Logs With a Compact Convolutional Transformer IEEE Access Anomaly detection deep learning self-supervised learning transformer |
| title | Detecting Anomalies in System Logs With a Compact Convolutional Transformer |
| title_full | Detecting Anomalies in System Logs With a Compact Convolutional Transformer |
| title_fullStr | Detecting Anomalies in System Logs With a Compact Convolutional Transformer |
| title_full_unstemmed | Detecting Anomalies in System Logs With a Compact Convolutional Transformer |
| title_short | Detecting Anomalies in System Logs With a Compact Convolutional Transformer |
| title_sort | detecting anomalies in system logs with a compact convolutional transformer |
| topic | Anomaly detection deep learning self-supervised learning transformer |
| url | https://ieeexplore.ieee.org/document/10285328/ |
| work_keys_str_mv | AT renelarisch detectinganomaliesinsystemlogswithacompactconvolutionaltransformer AT julienvitay detectinganomaliesinsystemlogswithacompactconvolutionaltransformer AT fredhhamker detectinganomaliesinsystemlogswithacompactconvolutionaltransformer |