Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm
The availability of an enormous amount of unlabeled datasets drives the anomaly detection research towards unsupervised machine learning algorithms. Deep clustering algorithms for anomaly detection gain significant research attention in this era. We propose an intelligent anomaly detection for exten...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9383226/ |
| _version_ | 1818933754180141056 |
|---|---|
| author | Annie Gilda Roselin Priyadarsi Nanda Surya Nepal Xiangjian He |
| author_facet | Annie Gilda Roselin Priyadarsi Nanda Surya Nepal Xiangjian He |
| author_sort | Annie Gilda Roselin |
| collection | DOAJ |
| description | The availability of an enormous amount of unlabeled datasets drives the anomaly detection research towards unsupervised machine learning algorithms. Deep clustering algorithms for anomaly detection gain significant research attention in this era. We propose an intelligent anomaly detection for extensive network traffic analysis with an Optimized Deep Clustering (ODC) algorithm. Firstly, ODC does the optimization of the deep AutoEncoder algorithm by tuning the hyperparameters. Thereby we can achieve a reduced reconstruction error rate from the deep AutoEncoder. Secondly, ODC feeds the optimized deep AutoEncoder’s latent view to the BIRCH clustering algorithm to detect the known and unknown malicious network traffic without human intervention. Unlike other deep clustering algorithms, ODC does not require to specify the number of clusters needed to analyze the network traffic dataset. We experiment ODC algorithm with the CoAP off-path dataset obtained from our testbed and the MNIST dataset to compare our algorithm’s accuracy with state-of-art clustering algorithms. The evaluation results show ODC deep clustering method outperforms the existing deep clustering methods for anomaly detection. |
| first_indexed | 2024-12-20T04:53:24Z |
| format | Article |
| id | doaj.art-a3988257e74049f49b032f67599aa12b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T04:53:24Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a3988257e74049f49b032f67599aa12b2022-12-21T19:52:47ZengIEEEIEEE Access2169-35362021-01-019472434725110.1109/ACCESS.2021.30681729383226Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) AlgorithmAnnie Gilda Roselin0https://orcid.org/0000-0003-0918-5104Priyadarsi Nanda1https://orcid.org/0000-0002-5748-155XSurya Nepal2https://orcid.org/0000-0002-3289-6599Xiangjian He3https://orcid.org/0000-0001-8962-540XDepartment of Electrical and Data Engineering, University of Technology Sydney (UTS), Ultimo, NSW, AustraliaDepartment of Electrical and Data Engineering, University of Technology Sydney (UTS), Ultimo, NSW, AustraliaCommonwealth Scientific and Industrial Research Organisation (CSIRO/Data61), Marsfield, NSW, AustraliaDepartment of Electrical and Data Engineering, University of Technology Sydney (UTS), Ultimo, NSW, AustraliaThe availability of an enormous amount of unlabeled datasets drives the anomaly detection research towards unsupervised machine learning algorithms. Deep clustering algorithms for anomaly detection gain significant research attention in this era. We propose an intelligent anomaly detection for extensive network traffic analysis with an Optimized Deep Clustering (ODC) algorithm. Firstly, ODC does the optimization of the deep AutoEncoder algorithm by tuning the hyperparameters. Thereby we can achieve a reduced reconstruction error rate from the deep AutoEncoder. Secondly, ODC feeds the optimized deep AutoEncoder’s latent view to the BIRCH clustering algorithm to detect the known and unknown malicious network traffic without human intervention. Unlike other deep clustering algorithms, ODC does not require to specify the number of clusters needed to analyze the network traffic dataset. We experiment ODC algorithm with the CoAP off-path dataset obtained from our testbed and the MNIST dataset to compare our algorithm’s accuracy with state-of-art clustering algorithms. The evaluation results show ODC deep clustering method outperforms the existing deep clustering methods for anomaly detection.https://ieeexplore.ieee.org/document/9383226/Deep learningAutoEncoderslatent space viewanomaly detectionregularizationBIRCH clustering |
| spellingShingle | Annie Gilda Roselin Priyadarsi Nanda Surya Nepal Xiangjian He Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm IEEE Access Deep learning AutoEncoders latent space view anomaly detection regularization BIRCH clustering |
| title | Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm |
| title_full | Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm |
| title_fullStr | Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm |
| title_full_unstemmed | Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm |
| title_short | Intelligent Anomaly Detection for Large Network Traffic With Optimized Deep Clustering (ODC) Algorithm |
| title_sort | intelligent anomaly detection for large network traffic with optimized deep clustering odc algorithm |
| topic | Deep learning AutoEncoders latent space view anomaly detection regularization BIRCH clustering |
| url | https://ieeexplore.ieee.org/document/9383226/ |
| work_keys_str_mv | AT anniegildaroselin intelligentanomalydetectionforlargenetworktrafficwithoptimizeddeepclusteringodcalgorithm AT priyadarsinanda intelligentanomalydetectionforlargenetworktrafficwithoptimizeddeepclusteringodcalgorithm AT suryanepal intelligentanomalydetectionforlargenetworktrafficwithoptimizeddeepclusteringodcalgorithm AT xiangjianhe intelligentanomalydetectionforlargenetworktrafficwithoptimizeddeepclusteringodcalgorithm |