Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis

Rootkits are malicious programs designed to conceal their activities on compromised systems, making them challenging to detect using conventional methods. As the threat landscape continually evolves, rootkits pose a serious threat by stealthily concealing malicious activities, making their early det...

Full description

Bibliographic Details
Main Authors: Basirah Noor, Sana Qadir
Format: Article
Language:English
Published: MDPI AG 2023-09-01
Series:Applied Sciences
Subjects:
Online Access:https://www.mdpi.com/2076-3417/13/19/10730
_version_ 1797576258108784640
author Basirah Noor
Sana Qadir
author_facet Basirah Noor
Sana Qadir
author_sort Basirah Noor
collection DOAJ
description Rootkits are malicious programs designed to conceal their activities on compromised systems, making them challenging to detect using conventional methods. As the threat landscape continually evolves, rootkits pose a serious threat by stealthily concealing malicious activities, making their early detection crucial to prevent data breaches and system compromise. A promising strategy for monitoring system activities involves analyzing volatile memory. This study proposes a rootkit detection model that combines memory analysis with Machine Learning (ML) and Deep Learning (DL) techniques. The model aims to identify suspicious patterns and behaviors associated with rootkits by analyzing the contents of a system’s volatile memory. To train the model, a diverse dataset of known rootkit samples is employed, and ML and deep learning algorithms are utilized. Through extensive experimentation and evaluation using SVM, RF, DT, k-NN, and LSTM algorithms, it is determined that SVM achieves the highest accuracy rate of 96.2%, whereas Execution Time (ET) shows that k-NN depicts the best performance, and LSTM (a DL model) shows the worst performance among the tested algorithms. This research contributes to the development of advanced defense mechanisms and enhances system security against the constantly evolving threat of rootkit attacks.
first_indexed 2024-03-10T21:49:44Z
format Article
id doaj.art-9ab90b7e792247ce941349f4db2fe469
institution Directory Open Access Journal
issn 2076-3417
language English
last_indexed 2024-03-10T21:49:44Z
publishDate 2023-09-01
publisher MDPI AG
record_format Article
series Applied Sciences
spelling doaj.art-9ab90b7e792247ce941349f4db2fe4692023-11-19T14:03:25ZengMDPI AGApplied Sciences2076-34172023-09-0113191073010.3390/app131910730Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory AnalysisBasirah Noor0Sana Qadir1School of Electrical Engineering and Computer Science, National University of Sciences and Technology, Islamabad 44000, PakistanSchool of Electrical Engineering and Computer Science, National University of Sciences and Technology, Islamabad 44000, PakistanRootkits are malicious programs designed to conceal their activities on compromised systems, making them challenging to detect using conventional methods. As the threat landscape continually evolves, rootkits pose a serious threat by stealthily concealing malicious activities, making their early detection crucial to prevent data breaches and system compromise. A promising strategy for monitoring system activities involves analyzing volatile memory. This study proposes a rootkit detection model that combines memory analysis with Machine Learning (ML) and Deep Learning (DL) techniques. The model aims to identify suspicious patterns and behaviors associated with rootkits by analyzing the contents of a system’s volatile memory. To train the model, a diverse dataset of known rootkit samples is employed, and ML and deep learning algorithms are utilized. Through extensive experimentation and evaluation using SVM, RF, DT, k-NN, and LSTM algorithms, it is determined that SVM achieves the highest accuracy rate of 96.2%, whereas Execution Time (ET) shows that k-NN depicts the best performance, and LSTM (a DL model) shows the worst performance among the tested algorithms. This research contributes to the development of advanced defense mechanisms and enhances system security against the constantly evolving threat of rootkit attacks.https://www.mdpi.com/2076-3417/13/19/10730memory analysisrootkitsdeep learningmachine learningexecution time
spellingShingle Basirah Noor
Sana Qadir
Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
Applied Sciences
memory analysis
rootkits
deep learning
machine learning
execution time
title Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
title_full Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
title_fullStr Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
title_full_unstemmed Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
title_short Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis
title_sort machine learning and deep learning based model for the detection of rootkits using memory analysis
topic memory analysis
rootkits
deep learning
machine learning
execution time
url https://www.mdpi.com/2076-3417/13/19/10730
work_keys_str_mv AT basirahnoor machinelearninganddeeplearningbasedmodelforthedetectionofrootkitsusingmemoryanalysis
AT sanaqadir machinelearninganddeeplearningbasedmodelforthedetectionofrootkitsusingmemoryanalysis