An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs
We perform an in-depth, systematic benchmarking study and evaluation of phishing features on diverse and extensive datasets. We propose a new taxonomy of features based on the interpretation and purpose of each feature. Next, we propose a benchmarking framework called `PhishBench,' which enable...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8970564/ |
| _version_ | 1818910230853976064 |
|---|---|
| author | Ayman El Aassal Shahryar Baki Avisha Das Rakesh M. Verma |
| author_facet | Ayman El Aassal Shahryar Baki Avisha Das Rakesh M. Verma |
| author_sort | Ayman El Aassal |
| collection | DOAJ |
| description | We perform an in-depth, systematic benchmarking study and evaluation of phishing features on diverse and extensive datasets. We propose a new taxonomy of features based on the interpretation and purpose of each feature. Next, we propose a benchmarking framework called `PhishBench,' which enables us to evaluate and compare the existing features for phishing detection systematically and thoroughly under identical experimental conditions, i.e., unified system specification, datasets, classifiers, and evaluation metrics. PhishBench is a first in the field of benchmarking phishing related research and incorporates thorough and systematic evaluation and feature comparison. We use PhishBench to test methods published in the phishing literature on new and diverse datasets to check their robustness and scalability. We study how dataset characteristics, e.g., varying legitimate to phishing ratios and increasing the size of imbalanced datasets, affect classification performance. Our results show that the imbalanced nature of phishing attacks affects the detection systems' performance and researchers should take this into account when proposing a new method. We also found that retraining alone is not enough to defeat new attacks. New features and techniques are required to stop attackers from fooling detection systems. |
| first_indexed | 2024-12-19T22:39:30Z |
| format | Article |
| id | doaj.art-0486f29d81da4ad7a0ea2e91fbcedaa7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T22:39:30Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-0486f29d81da4ad7a0ea2e91fbcedaa72022-12-21T20:03:07ZengIEEEIEEE Access2169-35362020-01-018221702219210.1109/ACCESS.2020.29697808970564An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security NeedsAyman El Aassal0Shahryar Baki1https://orcid.org/0000-0002-9814-9270Avisha Das2Rakesh M. Verma3Department of Computer Science, University of Houston, Houston, TX, USADepartment of Computer Science, University of Houston, Houston, TX, USADepartment of Computer Science, University of Houston, Houston, TX, USADepartment of Computer Science, University of Houston, Houston, TX, USAWe perform an in-depth, systematic benchmarking study and evaluation of phishing features on diverse and extensive datasets. We propose a new taxonomy of features based on the interpretation and purpose of each feature. Next, we propose a benchmarking framework called `PhishBench,' which enables us to evaluate and compare the existing features for phishing detection systematically and thoroughly under identical experimental conditions, i.e., unified system specification, datasets, classifiers, and evaluation metrics. PhishBench is a first in the field of benchmarking phishing related research and incorporates thorough and systematic evaluation and feature comparison. We use PhishBench to test methods published in the phishing literature on new and diverse datasets to check their robustness and scalability. We study how dataset characteristics, e.g., varying legitimate to phishing ratios and increasing the size of imbalanced datasets, affect classification performance. Our results show that the imbalanced nature of phishing attacks affects the detection systems' performance and researchers should take this into account when proposing a new method. We also found that retraining alone is not enough to defeat new attacks. New features and techniques are required to stop attackers from fooling detection systems.https://ieeexplore.ieee.org/document/8970564/Feature engineeringfeature taxonomyframeworkphishing emailphishing URLphishing website |
| spellingShingle | Ayman El Aassal Shahryar Baki Avisha Das Rakesh M. Verma An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs IEEE Access Feature engineering feature taxonomy framework phishing email phishing URL phishing website |
| title | An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs |
| title_full | An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs |
| title_fullStr | An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs |
| title_full_unstemmed | An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs |
| title_short | An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs |
| title_sort | in depth benchmarking and evaluation of phishing detection research for security needs |
| topic | Feature engineering feature taxonomy framework phishing email phishing URL phishing website |
| url | https://ieeexplore.ieee.org/document/8970564/ |
| work_keys_str_mv | AT aymanelaassal anindepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT shahryarbaki anindepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT avishadas anindepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT rakeshmverma anindepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT aymanelaassal indepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT shahryarbaki indepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT avishadas indepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds AT rakeshmverma indepthbenchmarkingandevaluationofphishingdetectionresearchforsecurityneeds |