System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals
System logs play an important role in software development and system maintenance. Many system software programs continuously generate system logs during software runtimes for failure detection and diagnosis purposes. Currently, the analysis of system log data is mainly a manual process that highly...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9709226/ |
| _version_ | 1818973382254788608 |
|---|---|
| author | Yixin Huangfu Saeid Habibi Alan Wassyng |
| author_facet | Yixin Huangfu Saeid Habibi Alan Wassyng |
| author_sort | Yixin Huangfu |
| collection | DOAJ |
| description | System logs play an important role in software development and system maintenance. Many system software programs continuously generate system logs during software runtimes for failure detection and diagnosis purposes. Currently, the analysis of system log data is mainly a manual process that highly depends on human knowledge and experience. This time-consuming task has become a problem because of the ever-increasing volume of log data. Existing studies have investigated machine learning and deep learning techniques to automate the failure detection task. This paper takes the deep learning approach and proposes two detection structures based on recurrent and convolutional neural networks. More importantly, this paper takes a step further by closely examining the timestamps of log data which existing studies have generally ignored. This study found that time information can be a distinguishing factor between regular and abnormal log sequences. Inspired by this observation, a novel method is proposed to integrate log timestamps in deep learning models using interpolation techniques. The evaluation results show that the log timestamps can significantly improve the performance of failure detection. Cross-comparison of the different models demonstrates that the proposed network structure can successfully utilize the timestamp information. The code is available on GitHub: <uri>https://github.com/hfyxin/Ts-models-log-data-analysis.git</uri>. |
| first_indexed | 2024-12-20T15:23:16Z |
| format | Article |
| id | doaj.art-08db0d2092ce41ae9a7e0ac7fd720ef5 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T15:23:16Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-08db0d2092ce41ae9a7e0ac7fd720ef52022-12-21T19:35:57ZengIEEEIEEE Access2169-35362022-01-0110176291764010.1109/ACCESS.2022.31503429709226System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform IntervalsYixin Huangfu0https://orcid.org/0000-0002-8635-7152Saeid Habibi1Alan Wassyng2https://orcid.org/0000-0003-4614-3421Department of Mechanical Engineering, McMaster University, Hamilton, ON, CanadaDepartment of Mechanical Engineering, McMaster University, Hamilton, ON, CanadaDepartment of Computing and Software, McMaster University, Hamilton, ON, CanadaSystem logs play an important role in software development and system maintenance. Many system software programs continuously generate system logs during software runtimes for failure detection and diagnosis purposes. Currently, the analysis of system log data is mainly a manual process that highly depends on human knowledge and experience. This time-consuming task has become a problem because of the ever-increasing volume of log data. Existing studies have investigated machine learning and deep learning techniques to automate the failure detection task. This paper takes the deep learning approach and proposes two detection structures based on recurrent and convolutional neural networks. More importantly, this paper takes a step further by closely examining the timestamps of log data which existing studies have generally ignored. This study found that time information can be a distinguishing factor between regular and abnormal log sequences. Inspired by this observation, a novel method is proposed to integrate log timestamps in deep learning models using interpolation techniques. The evaluation results show that the log timestamps can significantly improve the performance of failure detection. Cross-comparison of the different models demonstrates that the proposed network structure can successfully utilize the timestamp information. The code is available on GitHub: <uri>https://github.com/hfyxin/Ts-models-log-data-analysis.git</uri>.https://ieeexplore.ieee.org/document/9709226/Data engineeringdata miningfeature extractionneural networkspattern recognitionsoftware maintenance |
| spellingShingle | Yixin Huangfu Saeid Habibi Alan Wassyng System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals IEEE Access Data engineering data mining feature extraction neural networks pattern recognition software maintenance |
| title | System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals |
| title_full | System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals |
| title_fullStr | System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals |
| title_full_unstemmed | System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals |
| title_short | System Failure Detection Using Deep Learning Models Integrating Timestamps With Nonuniform Intervals |
| title_sort | system failure detection using deep learning models integrating timestamps with nonuniform intervals |
| topic | Data engineering data mining feature extraction neural networks pattern recognition software maintenance |
| url | https://ieeexplore.ieee.org/document/9709226/ |
| work_keys_str_mv | AT yixinhuangfu systemfailuredetectionusingdeeplearningmodelsintegratingtimestampswithnonuniformintervals AT saeidhabibi systemfailuredetectionusingdeeplearningmodelsintegratingtimestampswithnonuniformintervals AT alanwassyng systemfailuredetectionusingdeeplearningmodelsintegratingtimestampswithnonuniformintervals |