Measuring Early Detection of Anomalies
Early detection is a matter of growing importance in multiple domains as network security, health conditions over social network services or weather forecasts related disasters. It is not enough to make a good decision but it also needs to be made on time. In this paper, we define a method to evalua...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9963563/ |
| _version_ | 1811184610384543744 |
|---|---|
| author | Manuel F. Lopez-Vizcaino Francisco J. Novoa Diego Fernandez Fidel Cacheda |
| author_facet | Manuel F. Lopez-Vizcaino Francisco J. Novoa Diego Fernandez Fidel Cacheda |
| author_sort | Manuel F. Lopez-Vizcaino |
| collection | DOAJ |
| description | Early detection is a matter of growing importance in multiple domains as network security, health conditions over social network services or weather forecasts related disasters. It is not enough to make a good decision but it also needs to be made on time. In this paper, we define a method to evaluate detection of anomalies in time-aware systems. To do so, we present the early detection problem from a generic perspective, examine the evaluation metrics available and propose a new metric, named TaP (Time aware Precision). A set of experiments using three different datasets from different fields are performed in order to compare the behaviour of the different metrics. Two different approaches were followed, first a batch evaluation is performed, followed by a streaming evaluation which allows to present a more realistic behaviour of the systems. For both steps, we propose two sets of experiments. The first one using baseline models, followed by the evaluation of a set of Machine Learning algorithms results. The presented metric allows the amount of items needed to take a decision to be taken into account, not depending on the specific dataset but on the nature of the problem to solve. |
| first_indexed | 2024-04-11T13:15:14Z |
| format | Article |
| id | doaj.art-d1d7da9827a147df9569f990f9630ba1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T13:15:14Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d1d7da9827a147df9569f990f9630ba12022-12-22T04:22:25ZengIEEEIEEE Access2169-35362022-01-011012769512770710.1109/ACCESS.2022.32244679963563Measuring Early Detection of AnomaliesManuel F. Lopez-Vizcaino0https://orcid.org/0000-0001-8979-9880Francisco J. Novoa1https://orcid.org/0000-0003-3629-8120Diego Fernandez2https://orcid.org/0000-0002-6577-8951Fidel Cacheda3https://orcid.org/0000-0002-6438-1661Department of Computer Science and Information Technologies, Center for Information and Communications Technologies Research (CITIC), University of A Coruña, A Coruña, SpainDepartment of Computer Science and Information Technologies, Center for Information and Communications Technologies Research (CITIC), University of A Coruña, A Coruña, SpainDepartment of Computer Science and Information Technologies, Center for Information and Communications Technologies Research (CITIC), University of A Coruña, A Coruña, SpainDepartment of Computer Science and Information Technologies, Center for Information and Communications Technologies Research (CITIC), University of A Coruña, A Coruña, SpainEarly detection is a matter of growing importance in multiple domains as network security, health conditions over social network services or weather forecasts related disasters. It is not enough to make a good decision but it also needs to be made on time. In this paper, we define a method to evaluate detection of anomalies in time-aware systems. To do so, we present the early detection problem from a generic perspective, examine the evaluation metrics available and propose a new metric, named TaP (Time aware Precision). A set of experiments using three different datasets from different fields are performed in order to compare the behaviour of the different metrics. Two different approaches were followed, first a batch evaluation is performed, followed by a streaming evaluation which allows to present a more realistic behaviour of the systems. For both steps, we propose two sets of experiments. The first one using baseline models, followed by the evaluation of a set of Machine Learning algorithms results. The presented metric allows the amount of items needed to take a decision to be taken into account, not depending on the specific dataset but on the nature of the problem to solve.https://ieeexplore.ieee.org/document/9963563/Early detectionmachine learningtime-aware metricsreal-time systemsclassification algorithmsnetwork security |
| spellingShingle | Manuel F. Lopez-Vizcaino Francisco J. Novoa Diego Fernandez Fidel Cacheda Measuring Early Detection of Anomalies IEEE Access Early detection machine learning time-aware metrics real-time systems classification algorithms network security |
| title | Measuring Early Detection of Anomalies |
| title_full | Measuring Early Detection of Anomalies |
| title_fullStr | Measuring Early Detection of Anomalies |
| title_full_unstemmed | Measuring Early Detection of Anomalies |
| title_short | Measuring Early Detection of Anomalies |
| title_sort | measuring early detection of anomalies |
| topic | Early detection machine learning time-aware metrics real-time systems classification algorithms network security |
| url | https://ieeexplore.ieee.org/document/9963563/ |
| work_keys_str_mv | AT manuelflopezvizcaino measuringearlydetectionofanomalies AT franciscojnovoa measuringearlydetectionofanomalies AT diegofernandez measuringearlydetectionofanomalies AT fidelcacheda measuringearlydetectionofanomalies |