Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers
Security attacks on intelligent transportation systems (ITS) may result in life-threatening situations. Combining deep neural networks with reinforcement learning (RL) models called DRL shows promising results when applied to urban Traffic Signal Control (TSC) for adaptive adjustment of traffic ligh...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Open Journal of Intelligent Transportation Systems |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9566311/ |
| _version_ | 1818742509290913792 |
|---|---|
| author | Ammar Haydari Michael Zhang Chen-Nee Chuah |
| author_facet | Ammar Haydari Michael Zhang Chen-Nee Chuah |
| author_sort | Ammar Haydari |
| collection | DOAJ |
| description | Security attacks on intelligent transportation systems (ITS) may result in life-threatening situations. Combining deep neural networks with reinforcement learning (RL) models called DRL shows promising results when applied to urban Traffic Signal Control (TSC) for adaptive adjustment of traffic light schedules. In this paper, first, we explore the security vulnerabilities of DRL-based TSCs in the presence of adversarial attacks. We investigate the impact of the two distinct threat models with two state-of-the-art adversarial attacks using white-box and black-box settings. The attacks are simulated on different DRL-based TSC algorithms in a single intersection and multiple intersections. The results show that the performance of the DRL learning agent decreases in both adversarial attack models with white-box and black-box settings resulting in higher levels of traffic congestion. After analysing the adversarial attack models, we explored several sequential anomaly detection models. While sequential anomaly detection models minimizes the detection delays, it also achieves lower false alarm rates due to cumulative anomaly inspection. We also proposed an ensemble model that works with all the attack models without any model assumption. The results of anomaly detectors indicates that low-cost ensemble model achieves the best anomaly detection performance in all attack models and DRL settings. |
| first_indexed | 2024-12-18T02:13:39Z |
| format | Article |
| id | doaj.art-8d3faf61233841d5a2a4b75167c8d06b |
| institution | Directory Open Access Journal |
| issn | 2687-7813 |
| language | English |
| last_indexed | 2024-12-18T02:13:39Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Intelligent Transportation Systems |
| spelling | doaj.art-8d3faf61233841d5a2a4b75167c8d06b2022-12-21T21:24:26ZengIEEEIEEE Open Journal of Intelligent Transportation Systems2687-78132021-01-01240241610.1109/OJITS.2021.31189729566311Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal ControllersAmmar Haydari0https://orcid.org/0000-0003-1986-5656Michael Zhang1https://orcid.org/0000-0002-4647-3888Chen-Nee Chuah2https://orcid.org/0000-0002-2772-387XDepartment of Electrical and Computer Engineering, University of California at Davis, Davis, CA, USADepartment of Civil and Environmental Engineering, University of California at Davis, Davis, CA, USADepartment of Electrical and Computer Engineering, University of California at Davis, Davis, CA, USASecurity attacks on intelligent transportation systems (ITS) may result in life-threatening situations. Combining deep neural networks with reinforcement learning (RL) models called DRL shows promising results when applied to urban Traffic Signal Control (TSC) for adaptive adjustment of traffic light schedules. In this paper, first, we explore the security vulnerabilities of DRL-based TSCs in the presence of adversarial attacks. We investigate the impact of the two distinct threat models with two state-of-the-art adversarial attacks using white-box and black-box settings. The attacks are simulated on different DRL-based TSC algorithms in a single intersection and multiple intersections. The results show that the performance of the DRL learning agent decreases in both adversarial attack models with white-box and black-box settings resulting in higher levels of traffic congestion. After analysing the adversarial attack models, we explored several sequential anomaly detection models. While sequential anomaly detection models minimizes the detection delays, it also achieves lower false alarm rates due to cumulative anomaly inspection. We also proposed an ensemble model that works with all the attack models without any model assumption. The results of anomaly detectors indicates that low-cost ensemble model achieves the best anomaly detection performance in all attack models and DRL settings.https://ieeexplore.ieee.org/document/9566311/Deep reinforcement learningstatistical anomaly detectiontraffic signal controladversarial attacksecurity |
| spellingShingle | Ammar Haydari Michael Zhang Chen-Nee Chuah Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers IEEE Open Journal of Intelligent Transportation Systems Deep reinforcement learning statistical anomaly detection traffic signal control adversarial attack security |
| title | Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers |
| title_full | Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers |
| title_fullStr | Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers |
| title_full_unstemmed | Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers |
| title_short | Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers |
| title_sort | adversarial attacks and defense in deep reinforcement learning drl based traffic signal controllers |
| topic | Deep reinforcement learning statistical anomaly detection traffic signal control adversarial attack security |
| url | https://ieeexplore.ieee.org/document/9566311/ |
| work_keys_str_mv | AT ammarhaydari adversarialattacksanddefenseindeepreinforcementlearningdrlbasedtrafficsignalcontrollers AT michaelzhang adversarialattacksanddefenseindeepreinforcementlearningdrlbasedtrafficsignalcontrollers AT chenneechuah adversarialattacksanddefenseindeepreinforcementlearningdrlbasedtrafficsignalcontrollers |