Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices
The field of Natural Language Processing (NLP) is currently undergoing a revolutionary transformation driven by the power of pre-trained Large Language Models (LLMs) based on groundbreaking Transformer architectures. As the frequency and diversity of cybersecurity attacks continue to rise, the impor...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10423646/ |
| _version_ | 1797304890112868352 |
|---|---|
| author | Mohamed Amine Ferrag Mthandazo Ndhlovu Norbert Tihanyi Lucas C. Cordeiro Merouane Debbah Thierry Lestable Narinderjit Singh Thandi |
| author_facet | Mohamed Amine Ferrag Mthandazo Ndhlovu Norbert Tihanyi Lucas C. Cordeiro Merouane Debbah Thierry Lestable Narinderjit Singh Thandi |
| author_sort | Mohamed Amine Ferrag |
| collection | DOAJ |
| description | The field of Natural Language Processing (NLP) is currently undergoing a revolutionary transformation driven by the power of pre-trained Large Language Models (LLMs) based on groundbreaking Transformer architectures. As the frequency and diversity of cybersecurity attacks continue to rise, the importance of incident detection has significantly increased. IoT devices are expanding rapidly, resulting in a growing need for efficient techniques to autonomously identify network-based attacks in IoT networks with both high precision and minimal computational requirements. This paper presents SecurityBERT, a novel architecture that leverages the Bidirectional Encoder Representations from Transformers (BERT) model for cyber threat detection in IoT networks. During the training of SecurityBERT, we incorporated a novel privacy-preserving encoding technique called Privacy-Preserving Fixed-Length Encoding (PPFLE). We effectively represented network traffic data in a structured format by combining PPFLE with the Byte-level Byte-Pair Encoder (BBPE) Tokenizer. Our research demonstrates that SecurityBERT outperforms traditional Machine Learning (ML) and Deep Learning (DL) methods, such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), in cyber threat detection. Employing the Edge-IIoTset cybersecurity dataset, our experimental analysis shows that SecurityBERT achieved an impressive 98.2% overall accuracy in identifying fourteen distinct attack types, surpassing previous records set by hybrid solutions such as GAN-Transformer-based architectures and CNN-LSTM models. With an inference time of less than 0.15 seconds on an average CPU and a compact model size of just 16.7MB, SecurityBERT is ideally suited for real-life traffic analysis and a suitable choice for deployment on resource-constrained IoT devices. |
| first_indexed | 2024-03-08T00:16:55Z |
| format | Article |
| id | doaj.art-449290c1e12e415382c70527bb86aa84 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T00:16:55Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-449290c1e12e415382c70527bb86aa842024-02-17T00:02:12ZengIEEEIEEE Access2169-35362024-01-0112237332375010.1109/ACCESS.2024.336346910423646Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT DevicesMohamed Amine Ferrag0https://orcid.org/0000-0002-0632-3172Mthandazo Ndhlovu1https://orcid.org/0000-0003-0474-9548Norbert Tihanyi2https://orcid.org/0000-0002-9002-5935Lucas C. Cordeiro3https://orcid.org/0000-0002-6235-4272Merouane Debbah4https://orcid.org/0000-0001-8941-8080Thierry Lestable5Narinderjit Singh Thandi6Technology Innovation Institute, Abu Dhabi, United Arab EmiratesTechnology Innovation Institute, Abu Dhabi, United Arab EmiratesTechnology Innovation Institute, Abu Dhabi, United Arab EmiratesDepartment of Computer Science, The University of Manchester, Manchester, U.KKU 6G Research Center, Khalifa University of Science and Technology, Abu Dhabi, United Arab EmiratesTechnology Innovation Institute, Abu Dhabi, United Arab EmiratesTechnology Innovation Institute, Abu Dhabi, United Arab EmiratesThe field of Natural Language Processing (NLP) is currently undergoing a revolutionary transformation driven by the power of pre-trained Large Language Models (LLMs) based on groundbreaking Transformer architectures. As the frequency and diversity of cybersecurity attacks continue to rise, the importance of incident detection has significantly increased. IoT devices are expanding rapidly, resulting in a growing need for efficient techniques to autonomously identify network-based attacks in IoT networks with both high precision and minimal computational requirements. This paper presents SecurityBERT, a novel architecture that leverages the Bidirectional Encoder Representations from Transformers (BERT) model for cyber threat detection in IoT networks. During the training of SecurityBERT, we incorporated a novel privacy-preserving encoding technique called Privacy-Preserving Fixed-Length Encoding (PPFLE). We effectively represented network traffic data in a structured format by combining PPFLE with the Byte-level Byte-Pair Encoder (BBPE) Tokenizer. Our research demonstrates that SecurityBERT outperforms traditional Machine Learning (ML) and Deep Learning (DL) methods, such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), in cyber threat detection. Employing the Edge-IIoTset cybersecurity dataset, our experimental analysis shows that SecurityBERT achieved an impressive 98.2% overall accuracy in identifying fourteen distinct attack types, surpassing previous records set by hybrid solutions such as GAN-Transformer-based architectures and CNN-LSTM models. With an inference time of less than 0.15 seconds on an average CPU and a compact model size of just 16.7MB, SecurityBERT is ideally suited for real-life traffic analysis and a suitable choice for deployment on resource-constrained IoT devices.https://ieeexplore.ieee.org/document/10423646/Cyber threat detectionIoT networksgenerative AIBERTlarge language models |
| spellingShingle | Mohamed Amine Ferrag Mthandazo Ndhlovu Norbert Tihanyi Lucas C. Cordeiro Merouane Debbah Thierry Lestable Narinderjit Singh Thandi Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices IEEE Access Cyber threat detection IoT networks generative AI BERT large language models |
| title | Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices |
| title_full | Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices |
| title_fullStr | Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices |
| title_full_unstemmed | Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices |
| title_short | Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices |
| title_sort | revolutionizing cyber threat detection with large language models a privacy preserving bert based lightweight model for iot iiot devices |
| topic | Cyber threat detection IoT networks generative AI BERT large language models |
| url | https://ieeexplore.ieee.org/document/10423646/ |
| work_keys_str_mv | AT mohamedamineferrag revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT mthandazondhlovu revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT norberttihanyi revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT lucasccordeiro revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT merouanedebbah revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT thierrylestable revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices AT narinderjitsinghthandi revolutionizingcyberthreatdetectionwithlargelanguagemodelsaprivacypreservingbertbasedlightweightmodelforiotiiotdevices |