A two-stage intrusion detection method based on light gradient boosting machine and autoencoder
Intrusion detection systems can detect potential attacks and raise alerts on time. However, dimensionality curses and zero-day attacks pose challenges to intrusion detection systems. From a data perspective, the dimensionality curse leads to the low efficiency of intrusion detection systems. From th...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2023-02-01
|
| Series: | Mathematical Biosciences and Engineering |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/mbe.2023301?viewType=HTML |
| _version_ | 1811162821249990656 |
|---|---|
| author | Hao Zhang Lina Ge Guifen Zhang Jingwei Fan Denghui Li Chenyang Xu |
| author_facet | Hao Zhang Lina Ge Guifen Zhang Jingwei Fan Denghui Li Chenyang Xu |
| author_sort | Hao Zhang |
| collection | DOAJ |
| description | Intrusion detection systems can detect potential attacks and raise alerts on time. However, dimensionality curses and zero-day attacks pose challenges to intrusion detection systems. From a data perspective, the dimensionality curse leads to the low efficiency of intrusion detection systems. From the attack perspective, the increasing number of zero-day attacks overwhelms the intrusion detection system. To address these problems, this paper proposes a novel detection framework based on light gradient boosting machine (LightGBM) and autoencoder. The recursive feature elimination (RFE) method is first used for dimensionality reduction in this framework. Then a focal loss (FL) function is introduced into the LightGBM classifier to boost the learning of difficult samples. Finally, a two-stage prediction step with LightGBM and autoencoder is performed. In the first stage, pre-decision is conducted with LightGBM. In the second stage, a residual is used to make a secondary decision for samples with a normal class. The experiments were performed on the NSL-KDD and UNSWNB15 datasets, and compared with the classical method. It was found that the proposed method is superior to other methods and reduces the time overhead. In addition, the existing advanced methods were also compared in this study, and the results show that the proposed method is above 90% for accuracy, recall, and F1 score on both datasets. It is further concluded that our method is valid when compared with other advanced techniques. |
| first_indexed | 2024-04-10T06:35:17Z |
| format | Article |
| id | doaj.art-974973a2c2ac404db3c01b4dd4b515b9 |
| institution | Directory Open Access Journal |
| issn | 1551-0018 |
| language | English |
| last_indexed | 2024-04-10T06:35:17Z |
| publishDate | 2023-02-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | Mathematical Biosciences and Engineering |
| spelling | doaj.art-974973a2c2ac404db3c01b4dd4b515b92023-03-01T01:20:25ZengAIMS PressMathematical Biosciences and Engineering1551-00182023-02-012046966699210.3934/mbe.2023301A two-stage intrusion detection method based on light gradient boosting machine and autoencoderHao Zhang0Lina Ge1Guifen Zhang2Jingwei Fan 3Denghui Li 4Chenyang Xu51. School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China 2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, China1. School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China 2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, China 3. Guangxi Key Laboratory of Hybrid Computation and IC Design Analysis, Nanning 530006, China1. School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China 2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, China2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, China4. College of Electronic Information, Guangxi Minzu University, Nanning 530006, China1. School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China 2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, China1. School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China 2. Key Laboratory of Network Communication Engineering, Guangxi Minzu University, Nanning 530006, ChinaIntrusion detection systems can detect potential attacks and raise alerts on time. However, dimensionality curses and zero-day attacks pose challenges to intrusion detection systems. From a data perspective, the dimensionality curse leads to the low efficiency of intrusion detection systems. From the attack perspective, the increasing number of zero-day attacks overwhelms the intrusion detection system. To address these problems, this paper proposes a novel detection framework based on light gradient boosting machine (LightGBM) and autoencoder. The recursive feature elimination (RFE) method is first used for dimensionality reduction in this framework. Then a focal loss (FL) function is introduced into the LightGBM classifier to boost the learning of difficult samples. Finally, a two-stage prediction step with LightGBM and autoencoder is performed. In the first stage, pre-decision is conducted with LightGBM. In the second stage, a residual is used to make a secondary decision for samples with a normal class. The experiments were performed on the NSL-KDD and UNSWNB15 datasets, and compared with the classical method. It was found that the proposed method is superior to other methods and reduces the time overhead. In addition, the existing advanced methods were also compared in this study, and the results show that the proposed method is above 90% for accuracy, recall, and F1 score on both datasets. It is further concluded that our method is valid when compared with other advanced techniques.https://www.aimspress.com/article/doi/10.3934/mbe.2023301?viewType=HTMLcybersecurityfeature selectionfocal lossintrusion detection systemsmachine learning |
| spellingShingle | Hao Zhang Lina Ge Guifen Zhang Jingwei Fan Denghui Li Chenyang Xu A two-stage intrusion detection method based on light gradient boosting machine and autoencoder Mathematical Biosciences and Engineering cybersecurity feature selection focal loss intrusion detection systems machine learning |
| title | A two-stage intrusion detection method based on light gradient boosting machine and autoencoder |
| title_full | A two-stage intrusion detection method based on light gradient boosting machine and autoencoder |
| title_fullStr | A two-stage intrusion detection method based on light gradient boosting machine and autoencoder |
| title_full_unstemmed | A two-stage intrusion detection method based on light gradient boosting machine and autoencoder |
| title_short | A two-stage intrusion detection method based on light gradient boosting machine and autoencoder |
| title_sort | two stage intrusion detection method based on light gradient boosting machine and autoencoder |
| topic | cybersecurity feature selection focal loss intrusion detection systems machine learning |
| url | https://www.aimspress.com/article/doi/10.3934/mbe.2023301?viewType=HTML |
| work_keys_str_mv | AT haozhang atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT linage atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT guifenzhang atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT jingweifan atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT denghuili atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT chenyangxu atwostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT haozhang twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT linage twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT guifenzhang twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT jingweifan twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT denghuili twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder AT chenyangxu twostageintrusiondetectionmethodbasedonlightgradientboostingmachineandautoencoder |