On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code

Software protection from exploitation of possible unknown vulnerabilities can be performed both by searching (for example, using symbolic execution) and subsequent elimination of the vulnerabilities and by using detection and / or intrusion prevention systems. In the latter case, this problem is usu...

Full description

Bibliographic Details
Main Author: Yury V. Kosolapov
Format: Article
Language:English
Published: Yaroslavl State University 2020-06-01
Series:Моделирование и анализ информационных систем
Subjects:
Online Access:https://www.mais-journal.ru/jour/article/view/1321
_version_ 1797877821819846656
author Yury V. Kosolapov
author_facet Yury V. Kosolapov
author_sort Yury V. Kosolapov
collection DOAJ
description Software protection from exploitation of possible unknown vulnerabilities can be performed both by searching (for example, using symbolic execution) and subsequent elimination of the vulnerabilities and by using detection and / or intrusion prevention systems. In the latter case, this problem is usually solved by forming a profile of a normal behavior and deviation from normal behavior over a predetermined threshold is regarded as an anomaly or an attack. In this paper, the task is to protect a given software P from exploiting unknown vulnerabilities. For this aim a method is proposed for constructing a profile of the normal execution of the program P, in which, in addition to a set of legal chains of system and library functions, it is proposed to take into account the distances between adjacent function calls. At the same time, a profile is formed for each program. It is assumed that taking into account the distances between function calls will reveal shell code execution using system and / or library function calls. An algorithm and a system for detecting abnormal code execution are proposed. The work carried out experiments in the case when P is the FireFox browser. During the experiments the possibility of applying the developed algorithm to identify abnormal behavior when launching publicly available exploits was investigated.
first_indexed 2024-04-10T02:24:12Z
format Article
id doaj.art-9cde683028344141ad165253b430c966
institution Directory Open Access Journal
issn 1818-1015
2313-5417
language English
last_indexed 2024-04-10T02:24:12Z
publishDate 2020-06-01
publisher Yaroslavl State University
record_format Article
series Моделирование и анализ информационных систем
spelling doaj.art-9cde683028344141ad165253b430c9662023-03-13T08:07:35ZengYaroslavl State UniversityМоделирование и анализ информационных систем1818-10152313-54172020-06-0127213815110.18255/1818-1015-2020-2-138-151983On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious CodeYury V. Kosolapov0Южный Федеральный УниверситетSoftware protection from exploitation of possible unknown vulnerabilities can be performed both by searching (for example, using symbolic execution) and subsequent elimination of the vulnerabilities and by using detection and / or intrusion prevention systems. In the latter case, this problem is usually solved by forming a profile of a normal behavior and deviation from normal behavior over a predetermined threshold is regarded as an anomaly or an attack. In this paper, the task is to protect a given software P from exploiting unknown vulnerabilities. For this aim a method is proposed for constructing a profile of the normal execution of the program P, in which, in addition to a set of legal chains of system and library functions, it is proposed to take into account the distances between adjacent function calls. At the same time, a profile is formed for each program. It is assumed that taking into account the distances between function calls will reveal shell code execution using system and / or library function calls. An algorithm and a system for detecting abnormal code execution are proposed. The work carried out experiments in the case when P is the FireFox browser. During the experiments the possibility of applying the developed algorithm to identify abnormal behavior when launching publicly available exploits was investigated.https://www.mais-journal.ru/jour/article/view/1321системные вызовывызовы библиотекуязвимости программного обеспечения
spellingShingle Yury V. Kosolapov
On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
Моделирование и анализ информационных систем
системные вызовы
вызовы библиотек
уязвимости программного обеспечения
title On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
title_full On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
title_fullStr On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
title_full_unstemmed On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
title_short On the Detection of Exploitation of Vulnerabilities Leading to the Execution of a Malicious Code
title_sort on the detection of exploitation of vulnerabilities leading to the execution of a malicious code
topic системные вызовы
вызовы библиотек
уязвимости программного обеспечения
url https://www.mais-journal.ru/jour/article/view/1321
work_keys_str_mv AT yuryvkosolapov onthedetectionofexploitationofvulnerabilitiesleadingtotheexecutionofamaliciouscode