Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data
Major transportation surveillance protocols have not been specified with cyber security in mind and therefore provide no encryption nor identification. These issues expose air and sea transport to false data injection attacks (FDIAs), in which an attacker modifies, blocks or emits fake surveillance...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Proceedings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-3900/59/1/9 |
| _version_ | 1797546113599799296 |
|---|---|
| author | Antoine Chevrot Alexandre Vernotte Pierre Bernabe Aymeric Cretin Fabien Peureux Bruno Legeard |
| author_facet | Antoine Chevrot Alexandre Vernotte Pierre Bernabe Aymeric Cretin Fabien Peureux Bruno Legeard |
| author_sort | Antoine Chevrot |
| collection | DOAJ |
| description | Major transportation surveillance protocols have not been specified with cyber security in mind and therefore provide no encryption nor identification. These issues expose air and sea transport to false data injection attacks (FDIAs), in which an attacker modifies, blocks or emits fake surveillance messages to dupe controllers and surveillance systems. There has been growing interest in conducting research on machine learning-based anomaly detection systems that address these new threats. However, significant amounts of data are needed to achieve meaningful results with this type of model. Raw, genuine data can be obtained from existing databases but need to be preprocessed before being fed to a model. Acquiring anomalous data is another challenge: such data is much too scarce for both the Automatic Dependent Surveillance–Broadcast (ADS-B) and the Automatic Identification System (AIS). Crafting anomalous data by hand, which has been the sole method applied to date, is hardly suitable for broad detection model testing. This paper proposes an approach built upon existing libraries and ideas that offers ML researchers the necessary tools to facilitate the access and processing of genuine data as well as to automatically generate synthetic anomalous surveillance data to constitute broad, elaborated test datasets. We demonstrate the usability of the approach by discussing work in progress that includes the reproduction of related work, creation of relevant datasets and design of advanced anomaly detection models for both domains of application. |
| first_indexed | 2024-03-10T14:24:32Z |
| format | Article |
| id | doaj.art-a99963f2432c4494bc569cf5292ca9f3 |
| institution | Directory Open Access Journal |
| issn | 2504-3900 |
| language | English |
| last_indexed | 2024-03-10T14:24:32Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Proceedings |
| spelling | doaj.art-a99963f2432c4494bc569cf5292ca9f32023-11-20T23:05:19ZengMDPI AGProceedings2504-39002020-12-01591910.3390/proceedings2020059009Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance DataAntoine Chevrot0Alexandre Vernotte1Pierre Bernabe2Aymeric Cretin3Fabien Peureux4Bruno Legeard5FEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceFEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceFEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceFEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceFEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceFEMTO-ST Institute, University Bourgogne Franche-Comté, CNRS, 16 Route de Gray, 25030 Besançon, FranceMajor transportation surveillance protocols have not been specified with cyber security in mind and therefore provide no encryption nor identification. These issues expose air and sea transport to false data injection attacks (FDIAs), in which an attacker modifies, blocks or emits fake surveillance messages to dupe controllers and surveillance systems. There has been growing interest in conducting research on machine learning-based anomaly detection systems that address these new threats. However, significant amounts of data are needed to achieve meaningful results with this type of model. Raw, genuine data can be obtained from existing databases but need to be preprocessed before being fed to a model. Acquiring anomalous data is another challenge: such data is much too scarce for both the Automatic Dependent Surveillance–Broadcast (ADS-B) and the Automatic Identification System (AIS). Crafting anomalous data by hand, which has been the sole method applied to date, is hardly suitable for broad detection model testing. This paper proposes an approach built upon existing libraries and ideas that offers ML researchers the necessary tools to facilitate the access and processing of genuine data as well as to automatically generate synthetic anomalous surveillance data to constitute broad, elaborated test datasets. We demonstrate the usability of the approach by discussing work in progress that includes the reproduction of related work, creation of relevant datasets and design of advanced anomaly detection models for both domains of application.https://www.mdpi.com/2504-3900/59/1/9anomaly detectionsynthetic data generationAI testingATCVTS |
| spellingShingle | Antoine Chevrot Alexandre Vernotte Pierre Bernabe Aymeric Cretin Fabien Peureux Bruno Legeard Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data Proceedings anomaly detection synthetic data generation AI testing ATC VTS |
| title | Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data |
| title_full | Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data |
| title_fullStr | Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data |
| title_full_unstemmed | Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data |
| title_short | Improved Testing of AI-Based Anomaly Detection Systems Using Synthetic Surveillance Data |
| title_sort | improved testing of ai based anomaly detection systems using synthetic surveillance data |
| topic | anomaly detection synthetic data generation AI testing ATC VTS |
| url | https://www.mdpi.com/2504-3900/59/1/9 |
| work_keys_str_mv | AT antoinechevrot improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata AT alexandrevernotte improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata AT pierrebernabe improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata AT aymericcretin improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata AT fabienpeureux improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata AT brunolegeard improvedtestingofaibasedanomalydetectionsystemsusingsyntheticsurveillancedata |