An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems
This article presents a novel log abstraction framework based on neural open information extraction (OpenIE) and dynamic word embedding principles. Though various log parsing frameworks are proposed in the literature, the existing frameworks are modeled on predefined heuristics or auto-regressive me...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Open Journal of Instrumentation and Measurement |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10004508/ |
| _version_ | 1797197416746713088 |
|---|---|
| author | Susnata Bhattacharya Biplob Ray Ritesh Chugh Steven Gordon |
| author_facet | Susnata Bhattacharya Biplob Ray Ritesh Chugh Steven Gordon |
| author_sort | Susnata Bhattacharya |
| collection | DOAJ |
| description | This article presents a novel log abstraction framework based on neural open information extraction (OpenIE) and dynamic word embedding principles. Though various log parsing frameworks are proposed in the literature, the existing frameworks are modeled on predefined heuristics or auto-regressive methodologies that work well in offline scenarios. However, these frameworks are less suitable for dynamic self-adaptive systems, such as the Internet of Things (IoT), where the log outputs have diverse contextual variations and disparate time irregularities. Therefore, it is essential to move away from these traditional approaches and develop a systematic model that can effectively analyze log outputs in real-time and increase the system up-time of IoT networks so that they are almost always available. To address these needs, the proposed framework used OpenIE along with term frequency/inverse document frequency (TF/IDF) vectorization for constructing a set of relational triples (aka triple-sets). Additionally, a dynamic pretrained encoder–decoder architecture is utilized to imbibe the positional and contextualized information in its resultant outputs. The adopted methodology has enabled the proposed framework to extract richer word representations with dynamic contextualization of time-sensitive event logs to enhance further downstream activities, such as failure prediction and prognostic analysis of IoT networks. The proposed framework is evaluated on the system event log traces accumulated from a long range wide-area network (LoRaWAN) IoT gateway to proactively determine the probable causes of its various failure scenarios. Additionally, the study also provided a comparative analysis of its mathematical representations with that of the current state-of-the-art (SOTA) approaches to project the advantages and benefits of the proposed model, particularly from its data analytics standpoint. |
| first_indexed | 2024-04-24T06:43:37Z |
| format | Article |
| id | doaj.art-bf7741de78644882ada23d47de2b823a |
| institution | Directory Open Access Journal |
| issn | 2768-7236 |
| language | English |
| last_indexed | 2024-04-24T06:43:37Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Instrumentation and Measurement |
| spelling | doaj.art-bf7741de78644882ada23d47de2b823a2024-04-22T20:23:39ZengIEEEIEEE Open Journal of Instrumentation and Measurement2768-72362023-01-01211810.1109/OJIM.2022.323265010004508An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things SystemsSusnata Bhattacharya0https://orcid.org/0000-0002-8189-8435Biplob Ray1https://orcid.org/0000-0002-3016-1695Ritesh Chugh2https://orcid.org/0000-0003-0061-7206Steven Gordon3https://orcid.org/0000-0003-4090-1199School of Engineering and Technology, Central Queensland University, Rockhampton, QLD, AustraliaSchool of Engineering and Technology, Central Queensland University, Rockhampton, QLD, AustraliaSchool of Engineering and Technology, Central Queensland University, Rockhampton, QLD, AustraliaSchool of Engineering and Technology, Central Queensland University, Rockhampton, QLD, AustraliaThis article presents a novel log abstraction framework based on neural open information extraction (OpenIE) and dynamic word embedding principles. Though various log parsing frameworks are proposed in the literature, the existing frameworks are modeled on predefined heuristics or auto-regressive methodologies that work well in offline scenarios. However, these frameworks are less suitable for dynamic self-adaptive systems, such as the Internet of Things (IoT), where the log outputs have diverse contextual variations and disparate time irregularities. Therefore, it is essential to move away from these traditional approaches and develop a systematic model that can effectively analyze log outputs in real-time and increase the system up-time of IoT networks so that they are almost always available. To address these needs, the proposed framework used OpenIE along with term frequency/inverse document frequency (TF/IDF) vectorization for constructing a set of relational triples (aka triple-sets). Additionally, a dynamic pretrained encoder–decoder architecture is utilized to imbibe the positional and contextualized information in its resultant outputs. The adopted methodology has enabled the proposed framework to extract richer word representations with dynamic contextualization of time-sensitive event logs to enhance further downstream activities, such as failure prediction and prognostic analysis of IoT networks. The proposed framework is evaluated on the system event log traces accumulated from a long range wide-area network (LoRaWAN) IoT gateway to proactively determine the probable causes of its various failure scenarios. Additionally, the study also provided a comparative analysis of its mathematical representations with that of the current state-of-the-art (SOTA) approaches to project the advantages and benefits of the proposed model, particularly from its data analytics standpoint.https://ieeexplore.ieee.org/document/10004508/BERTcontext-awaregatewayinformation extractionInternet of Things (IoT)log parsing |
| spellingShingle | Susnata Bhattacharya Biplob Ray Ritesh Chugh Steven Gordon An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems IEEE Open Journal of Instrumentation and Measurement BERT context-aware gateway information extraction Internet of Things (IoT) log parsing |
| title | An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems |
| title_full | An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems |
| title_fullStr | An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems |
| title_full_unstemmed | An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems |
| title_short | An Online Parsing Framework for Semistructured Streaming System Logs of Internet of Things Systems |
| title_sort | online parsing framework for semistructured streaming system logs of internet of things systems |
| topic | BERT context-aware gateway information extraction Internet of Things (IoT) log parsing |
| url | https://ieeexplore.ieee.org/document/10004508/ |
| work_keys_str_mv | AT susnatabhattacharya anonlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT biplobray anonlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT riteshchugh anonlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT stevengordon anonlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT susnatabhattacharya onlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT biplobray onlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT riteshchugh onlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems AT stevengordon onlineparsingframeworkforsemistructuredstreamingsystemlogsofinternetofthingssystems |