A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments
Detecting anomalies in data streams from smart communication environments is a challenging problem that can benefit from novel learning techniques. The Attention Mechanism is a very promising architecture for addressing this problem. It allows the model to focus on specific parts of the input data w...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/17/9648 |
| _version_ | 1827728336569237504 |
|---|---|
| author | Konstantinos Demertzis Konstantinos Rantos Lykourgos Magafas Lazaros Iliadis |
| author_facet | Konstantinos Demertzis Konstantinos Rantos Lykourgos Magafas Lazaros Iliadis |
| author_sort | Konstantinos Demertzis |
| collection | DOAJ |
| description | Detecting anomalies in data streams from smart communication environments is a challenging problem that can benefit from novel learning techniques. The Attention Mechanism is a very promising architecture for addressing this problem. It allows the model to focus on specific parts of the input data when processing it, improving its ability to understand the meaning of specific parts in context and make more accurate predictions. This paper presents a Cross-Modal Dynamic Attention Neural Architecture (CM-DANA) by expanding on state-of-the-art techniques. It is a novel dynamic attention mechanism that can be trained end-to-end along with the rest of the model using multimodal data streams. The attention mechanism calculates attention weights for each position in the input data based on the model’s current state by a hybrid method called Cross-Modal Attention. Specifically, the proposed model uses multimodal learning tasks where the input data comes from different cyber modalities. It combines the relevant input data using these weights to produce an attention vector in order to detect suspicious abnormal behavior. We demonstrate the effectiveness of our approach on a cyber security anomalies detection task using multiple data streams from smart communication environments. |
| first_indexed | 2024-03-10T23:27:47Z |
| format | Article |
| id | doaj.art-8b23ad8ecaa04f9884d36e6fa36197b7 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T23:27:47Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-8b23ad8ecaa04f9884d36e6fa36197b72023-11-19T07:49:54ZengMDPI AGApplied Sciences2076-34172023-08-011317964810.3390/app13179648A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication EnvironmentsKonstantinos Demertzis0Konstantinos Rantos1Lykourgos Magafas2Lazaros Iliadis3Department of Computer Science, School of Science, International Hellenic University, 65404 Kavala, GreeceDepartment of Computer Science, School of Science, International Hellenic University, 65404 Kavala, GreeceDepartment of Physics, School of Science, Kavala Campus, International Hellenic University, 65404 Kavala, GreeceDepartment of Civil Engineering, School of Engineering, Democritus University of Thrace, 67100 Xanthi, GreeceDetecting anomalies in data streams from smart communication environments is a challenging problem that can benefit from novel learning techniques. The Attention Mechanism is a very promising architecture for addressing this problem. It allows the model to focus on specific parts of the input data when processing it, improving its ability to understand the meaning of specific parts in context and make more accurate predictions. This paper presents a Cross-Modal Dynamic Attention Neural Architecture (CM-DANA) by expanding on state-of-the-art techniques. It is a novel dynamic attention mechanism that can be trained end-to-end along with the rest of the model using multimodal data streams. The attention mechanism calculates attention weights for each position in the input data based on the model’s current state by a hybrid method called Cross-Modal Attention. Specifically, the proposed model uses multimodal learning tasks where the input data comes from different cyber modalities. It combines the relevant input data using these weights to produce an attention vector in order to detect suspicious abnormal behavior. We demonstrate the effectiveness of our approach on a cyber security anomalies detection task using multiple data streams from smart communication environments.https://www.mdpi.com/2076-3417/13/17/9648cross-modal learning tasksdynamic attention mechanismneural architectureanomaly detectiondata streamssmart communication environments |
| spellingShingle | Konstantinos Demertzis Konstantinos Rantos Lykourgos Magafas Lazaros Iliadis A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments Applied Sciences cross-modal learning tasks dynamic attention mechanism neural architecture anomaly detection data streams smart communication environments |
| title | A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments |
| title_full | A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments |
| title_fullStr | A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments |
| title_full_unstemmed | A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments |
| title_short | A Cross-Modal Dynamic Attention Neural Architecture to Detect Anomalies in Data Streams from Smart Communication Environments |
| title_sort | cross modal dynamic attention neural architecture to detect anomalies in data streams from smart communication environments |
| topic | cross-modal learning tasks dynamic attention mechanism neural architecture anomaly detection data streams smart communication environments |
| url | https://www.mdpi.com/2076-3417/13/17/9648 |
| work_keys_str_mv | AT konstantinosdemertzis acrossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT konstantinosrantos acrossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT lykourgosmagafas acrossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT lazarosiliadis acrossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT konstantinosdemertzis crossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT konstantinosrantos crossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT lykourgosmagafas crossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments AT lazarosiliadis crossmodaldynamicattentionneuralarchitecturetodetectanomaliesindatastreamsfromsmartcommunicationenvironments |