Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering
Novelty detection is a classification problem to identify abnormal patterns; therefore, it is an important task for applications such as fraud detection, fault diagnosis and disease detection. However, when there is no label that indicates normal and abnormal data, it will need expensive domain and...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | http://www.mdpi.com/2076-3417/8/9/1468 |
| _version_ | 1811331325128343552 |
|---|---|
| author | Tsatsral Amarbayasgalan Bilguun Jargalsaikhan Keun Ho Ryu |
| author_facet | Tsatsral Amarbayasgalan Bilguun Jargalsaikhan Keun Ho Ryu |
| author_sort | Tsatsral Amarbayasgalan |
| collection | DOAJ |
| description | Novelty detection is a classification problem to identify abnormal patterns; therefore, it is an important task for applications such as fraud detection, fault diagnosis and disease detection. However, when there is no label that indicates normal and abnormal data, it will need expensive domain and professional knowledge, so an unsupervised novelty detection approach will be used. On the other hand, nowadays, using novelty detection on high dimensional data is a big challenge and previous research suggests approaches based on principal component analysis (PCA) and an autoencoder in order to reduce dimensionality. In this paper, we propose deep autoencoders with density based clustering (DAE-DBC); this approach calculates compressed data and error threshold from deep autoencoder model, sending the results to a density based cluster. Points that are not involved in any groups are not considered a novelty; the grouping points will be defined as a novelty group depending on the ratio of the points exceeding the error threshold. We have conducted the experiment by substituting components to show that the components of the proposed method together are more effective. As a result of the experiment, the DAE-DBC approach is more efficient; its area under the curve (AUC) is shown to be 13.5 percent higher than state-of-the-art algorithms and other versions of the proposed method that we have demonstrated. |
| first_indexed | 2024-04-13T16:17:53Z |
| format | Article |
| id | doaj.art-22dd039c7670433bb5b4a2ddc8aae8a2 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-04-13T16:17:53Z |
| publishDate | 2018-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-22dd039c7670433bb5b4a2ddc8aae8a22022-12-22T02:40:00ZengMDPI AGApplied Sciences2076-34172018-08-0189146810.3390/app8091468app8091468Unsupervised Novelty Detection Using Deep Autoencoders with Density Based ClusteringTsatsral Amarbayasgalan0Bilguun Jargalsaikhan1Keun Ho Ryu2Database and Bioinformatics Laboratory, School of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, KoreaDatabase and Bioinformatics Laboratory, School of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, KoreaDatabase and Bioinformatics Laboratory, School of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, KoreaNovelty detection is a classification problem to identify abnormal patterns; therefore, it is an important task for applications such as fraud detection, fault diagnosis and disease detection. However, when there is no label that indicates normal and abnormal data, it will need expensive domain and professional knowledge, so an unsupervised novelty detection approach will be used. On the other hand, nowadays, using novelty detection on high dimensional data is a big challenge and previous research suggests approaches based on principal component analysis (PCA) and an autoencoder in order to reduce dimensionality. In this paper, we propose deep autoencoders with density based clustering (DAE-DBC); this approach calculates compressed data and error threshold from deep autoencoder model, sending the results to a density based cluster. Points that are not involved in any groups are not considered a novelty; the grouping points will be defined as a novelty group depending on the ratio of the points exceeding the error threshold. We have conducted the experiment by substituting components to show that the components of the proposed method together are more effective. As a result of the experiment, the DAE-DBC approach is more efficient; its area under the curve (AUC) is shown to be 13.5 percent higher than state-of-the-art algorithms and other versions of the proposed method that we have demonstrated.http://www.mdpi.com/2076-3417/8/9/1468novelty detectiondimensionality reductionclustering |
| spellingShingle | Tsatsral Amarbayasgalan Bilguun Jargalsaikhan Keun Ho Ryu Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering Applied Sciences novelty detection dimensionality reduction clustering |
| title | Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering |
| title_full | Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering |
| title_fullStr | Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering |
| title_full_unstemmed | Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering |
| title_short | Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering |
| title_sort | unsupervised novelty detection using deep autoencoders with density based clustering |
| topic | novelty detection dimensionality reduction clustering |
| url | http://www.mdpi.com/2076-3417/8/9/1468 |
| work_keys_str_mv | AT tsatsralamarbayasgalan unsupervisednoveltydetectionusingdeepautoencoderswithdensitybasedclustering AT bilguunjargalsaikhan unsupervisednoveltydetectionusingdeepautoencoderswithdensitybasedclustering AT keunhoryu unsupervisednoveltydetectionusingdeepautoencoderswithdensitybasedclustering |