A Neural Network-Based Approach for Cryptographic Function Detection in Malware

Cryptographic technology has been commonly used in malware for hiding their static characteristics and malicious behaviors to avoid the detection of anti-virus engines and counter the reverse analysis from security researchers. The detection of cryptographic functions in an effective way in malware...

Full description

Bibliographic Details
Main Authors: Li Jia, Anmin Zhou, Peng Jia, Luping Liu, Yan Wang, Liang Liu
Format: Article
Language:English
Published: IEEE 2020-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/8960303/
_version_ 1818910121248423936
author Li Jia
Anmin Zhou
Peng Jia
Luping Liu
Yan Wang
Liang Liu
author_facet Li Jia
Anmin Zhou
Peng Jia
Luping Liu
Yan Wang
Liang Liu
author_sort Li Jia
collection DOAJ
description Cryptographic technology has been commonly used in malware for hiding their static characteristics and malicious behaviors to avoid the detection of anti-virus engines and counter the reverse analysis from security researchers. The detection of cryptographic functions in an effective way in malware has vital significance for malicious code detection and deep analysis. Many efforts have been made to solve this issue, while existing methods suffer from some issues, such as unable to achieve promising results in accuracy, limited by prior knowledge, and have a high overhead. In this paper, we draw on the idea of text classification in the field of natural language processing and propose a novel neural network to detect the type of cryptographic functions. The new network is an end-2-end model which includes two important modules: Instruction-2-vec and K-Max-CNN-Attention. The Instruction-2-vec model extracts the “words” of assembly instructions and transfers them into continuous vectors. The K-Max-CNN-Attention is used to encode the instruction vectors and generate the representation of the function. And we designed a softmax classifier to predict the categories of the functions. Extensive experiments were conducted on a collected dataset which contains 15 common types of cryptographic functions extracted from malware, to assess the validity of the proposed approach. The experiment results showed that the proposed approach archives a better performance than the recent embedding network SAFE with the Precision, Recall and F1-score of 0.9349, 0.8933 and 0.9020, respectively. We also compared it with four widely-used tools, the results demonstrated that our approach is much better in accuracy and effectiveness than all of them.
first_indexed 2024-12-19T22:37:46Z
format Article
id doaj.art-7e45bbc3e4a74909a5d335af7614e67b
institution Directory Open Access Journal
issn 2169-3536
language English
last_indexed 2024-12-19T22:37:46Z
publishDate 2020-01-01
publisher IEEE
record_format Article
series IEEE Access
spelling doaj.art-7e45bbc3e4a74909a5d335af7614e67b2022-12-21T20:03:09ZengIEEEIEEE Access2169-35362020-01-018235062352110.1109/ACCESS.2020.29668608960303A Neural Network-Based Approach for Cryptographic Function Detection in MalwareLi Jia0https://orcid.org/0000-0001-8790-2135Anmin Zhou1Peng Jia2Luping Liu3https://orcid.org/0000-0002-2967-2682Yan Wang4Liang Liu5https://orcid.org/0000-0002-5612-1915College of Cybersecurity, Sichuan University, Chengdu, ChinaCollege of Cybersecurity, Sichuan University, Chengdu, ChinaCollege of Cybersecurity, Sichuan University, Chengdu, ChinaCollege of Electronics and Information Engineering, Sichuan University, Chengdu, ChinaCollege of Cybersecurity, Sichuan University, Chengdu, ChinaCollege of Cybersecurity, Sichuan University, Chengdu, ChinaCryptographic technology has been commonly used in malware for hiding their static characteristics and malicious behaviors to avoid the detection of anti-virus engines and counter the reverse analysis from security researchers. The detection of cryptographic functions in an effective way in malware has vital significance for malicious code detection and deep analysis. Many efforts have been made to solve this issue, while existing methods suffer from some issues, such as unable to achieve promising results in accuracy, limited by prior knowledge, and have a high overhead. In this paper, we draw on the idea of text classification in the field of natural language processing and propose a novel neural network to detect the type of cryptographic functions. The new network is an end-2-end model which includes two important modules: Instruction-2-vec and K-Max-CNN-Attention. The Instruction-2-vec model extracts the “words” of assembly instructions and transfers them into continuous vectors. The K-Max-CNN-Attention is used to encode the instruction vectors and generate the representation of the function. And we designed a softmax classifier to predict the categories of the functions. Extensive experiments were conducted on a collected dataset which contains 15 common types of cryptographic functions extracted from malware, to assess the validity of the proposed approach. The experiment results showed that the proposed approach archives a better performance than the recent embedding network SAFE with the Precision, Recall and F1-score of 0.9349, 0.8933 and 0.9020, respectively. We also compared it with four widely-used tools, the results demonstrated that our approach is much better in accuracy and effectiveness than all of them.https://ieeexplore.ieee.org/document/8960303/Cryptographic function detectionneural networkfunction embeddingbinary analysis
spellingShingle Li Jia
Anmin Zhou
Peng Jia
Luping Liu
Yan Wang
Liang Liu
A Neural Network-Based Approach for Cryptographic Function Detection in Malware
IEEE Access
Cryptographic function detection
neural network
function embedding
binary analysis
title A Neural Network-Based Approach for Cryptographic Function Detection in Malware
title_full A Neural Network-Based Approach for Cryptographic Function Detection in Malware
title_fullStr A Neural Network-Based Approach for Cryptographic Function Detection in Malware
title_full_unstemmed A Neural Network-Based Approach for Cryptographic Function Detection in Malware
title_short A Neural Network-Based Approach for Cryptographic Function Detection in Malware
title_sort neural network based approach for cryptographic function detection in malware
topic Cryptographic function detection
neural network
function embedding
binary analysis
url https://ieeexplore.ieee.org/document/8960303/
work_keys_str_mv AT lijia aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT anminzhou aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT pengjia aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT lupingliu aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT yanwang aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT liangliu aneuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT lijia neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT anminzhou neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT pengjia neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT lupingliu neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT yanwang neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware
AT liangliu neuralnetworkbasedapproachforcryptographicfunctiondetectioninmalware