Improving security of autonomous cyber-physical systems against adversarial examples
Deep learning, enabled by the advancements of hardware accelerators, is increasingly employed in cyber-physical systems due to its capabilities in capturing sophisticated patterns from complex physical processes. However, deep learning is shown susceptible to adversarial examples, which are crafted...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Doctor of Philosophy |
| Language: | English |
| Published: |
Nanyang Technological University
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/161165 |
| _version_ | 1811685810790989824 |
|---|---|
| author | Song, Qun |
| author2 | Tan Rui |
| author_facet | Tan Rui Song, Qun |
| author_sort | Song, Qun |
| collection | NTU |
| description | Deep learning, enabled by the advancements of hardware accelerators, is increasingly employed in cyber-physical systems due to its capabilities in capturing sophisticated patterns from complex physical processes. However, deep learning is shown susceptible to adversarial examples, which are crafted inputs aiming to cause wrong classification outputs for deep models by adding minute perturbations on the clean inputs. Thus, deploying deep learning models on the safety-critical cyber-physical systems without incorporating effective countermeasures against adversarial examples raises security concerns.
This thesis investigates the threat of adversarial example attack and develops effective defenses for two deep learning-based autonomous sensing tasks of cyber-physical systems, i.e., visual sensing of advanced driver assistance systems and drones as well as the voltage stability assessment of smart grids.
Deep learning achieves appealing performance in accurate and resilient perception of complex environments. Thus, deep models are increasingly adopted for visual sensing in autonomous systems such as vehicles and drones. However, it is shown that deep models are vulnerable to adversarial attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Most existing countermeasures against adversarial examples build their security on the attackers' ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs's principle and can be subverted once the attackers know the details of the defense. This thesis proposes DeepMTD, which applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment, that will collaboratively detect and thwart adversarial examples. The MTD design in DeepMTD is based on the adversarial examples' minor transferability across different models. The post-deployment of dynamically generated models significantly increases the bar of successful attacks. This thesis also investigates serial data fusion with early stopping for DeepMTD to reduce the inference time by a factor of up to 5, as well as exploits hardware inference accelerators' characteristics to strike better trade-offs between inference time and power consumption. Evaluation based on three datasets including a road sign dataset and two GPU-equipped embedded computing boards shows the effectiveness and efficiency of DeepMTD in counteracting the attack.
To further advance the MTD defense, this thesis presents Sardino, an active and dynamic defense approach that renews the inference ensemble at run time to develop security against the adaptive adversary who tries to exfiltrate the ensemble and construct the corresponding effective adversarial examples. By applying consistency check and data fusion on the ensemble's predictions, Sardino can detect and thwart adversarial inputs. Compared with the training-based ensemble renewal approach adopted by DeepMTD, Sardino uses HyperNet to achieve one million times acceleration and per-frame ensemble renewal that presents the highest level of difficulty to the prerequisite exfiltration attacks. This thesis designs a run-time planner for Sardino that maximizes the ensemble size in favor of security while maintaining the processing frame rate. Beyond adversarial examples, Sardino can also address the issue of out-of-distribution inputs effectively. This thesis presents extensive evaluation of Sardino's performance in counteracting adversarial examples and applies it to build a real-time car-borne traffic sign recognition system. Live on-road tests show the built system's effectiveness in maintaining frame rate and detecting out-of-distribution inputs due to the false positives of a preceding YOLO-based traffic sign detector.
Voltage stability assessment is essential for maintaining reliable power grid operations. Stability assessment approaches using deep learning address the shortfalls of the traditional time-domain simulation-based approaches caused by increased system complexity. However, deep learning models are shown vulnerable to adversarial examples in the field of computer vision. While this vulnerability has been noticed by the power grid cybersecurity research, the domain-specific analysis on the requirements imposed upon effective attack implementation is still lacking. Although these attack requirements are usually reasonable in computer vision tasks, they can be stringent in the context of power grids. This thesis conducts a systematic investigation on the attack requirements and credibility of six representative adversarial example attacks based on a voltage stability assessment application for the New England 10-machine 39-bus power system. The investigation in this thesis shows that (1) compromising about half the transmission system buses' voltage traces is a rule-of-thumb attack requirement; (2) the universal adversarial perturbations regardless of the original clean voltage trajectory possess the same credibility as the widely studied false data injection attacks on power grid state estimation, while the input-specific adversarial perturbations are less credible; (3) the prevailing strong adversarial training thwarts the universal perturbations but fails in defending certain input-specific perturbations. To advance defense to cope with both universal and input-specific adversarial examples, this thesis proposes a new approach that simultaneously estimates the predictive uncertainty of any given input of voltage trajectory and thwarts the attacks effectively.
To summarize, this thesis studies the threat and countermeasures for the adversarial example attack as an ongoing concern for the safety-critical autonomous cyber-physical systems. It develops DeepMTD and Sardino, which are two dynamic ensemble-based defenses designed under the strategy of moving target defense, to effectively counteract the adaptive adversarial example adversary for embedded deep visual sensing. It also conducts the systematic requirement investigation and credibility analysis of adversarial example attack against the power grid voltage stability assessment and develops effective countermeasure. |
| first_indexed | 2024-10-01T04:50:27Z |
| format | Thesis-Doctor of Philosophy |
| id | ntu-10356/161165 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:50:27Z |
| publishDate | 2022 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1611652023-03-05T16:34:27Z Improving security of autonomous cyber-physical systems against adversarial examples Song, Qun Tan Rui Interdisciplinary Graduate School (IGS) Energy Research Institute @ NTU (ERI@N) tanrui@ntu.edu.sg Engineering::Computer science and engineering Deep learning, enabled by the advancements of hardware accelerators, is increasingly employed in cyber-physical systems due to its capabilities in capturing sophisticated patterns from complex physical processes. However, deep learning is shown susceptible to adversarial examples, which are crafted inputs aiming to cause wrong classification outputs for deep models by adding minute perturbations on the clean inputs. Thus, deploying deep learning models on the safety-critical cyber-physical systems without incorporating effective countermeasures against adversarial examples raises security concerns. This thesis investigates the threat of adversarial example attack and develops effective defenses for two deep learning-based autonomous sensing tasks of cyber-physical systems, i.e., visual sensing of advanced driver assistance systems and drones as well as the voltage stability assessment of smart grids. Deep learning achieves appealing performance in accurate and resilient perception of complex environments. Thus, deep models are increasingly adopted for visual sensing in autonomous systems such as vehicles and drones. However, it is shown that deep models are vulnerable to adversarial attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Most existing countermeasures against adversarial examples build their security on the attackers' ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs's principle and can be subverted once the attackers know the details of the defense. This thesis proposes DeepMTD, which applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment, that will collaboratively detect and thwart adversarial examples. The MTD design in DeepMTD is based on the adversarial examples' minor transferability across different models. The post-deployment of dynamically generated models significantly increases the bar of successful attacks. This thesis also investigates serial data fusion with early stopping for DeepMTD to reduce the inference time by a factor of up to 5, as well as exploits hardware inference accelerators' characteristics to strike better trade-offs between inference time and power consumption. Evaluation based on three datasets including a road sign dataset and two GPU-equipped embedded computing boards shows the effectiveness and efficiency of DeepMTD in counteracting the attack. To further advance the MTD defense, this thesis presents Sardino, an active and dynamic defense approach that renews the inference ensemble at run time to develop security against the adaptive adversary who tries to exfiltrate the ensemble and construct the corresponding effective adversarial examples. By applying consistency check and data fusion on the ensemble's predictions, Sardino can detect and thwart adversarial inputs. Compared with the training-based ensemble renewal approach adopted by DeepMTD, Sardino uses HyperNet to achieve one million times acceleration and per-frame ensemble renewal that presents the highest level of difficulty to the prerequisite exfiltration attacks. This thesis designs a run-time planner for Sardino that maximizes the ensemble size in favor of security while maintaining the processing frame rate. Beyond adversarial examples, Sardino can also address the issue of out-of-distribution inputs effectively. This thesis presents extensive evaluation of Sardino's performance in counteracting adversarial examples and applies it to build a real-time car-borne traffic sign recognition system. Live on-road tests show the built system's effectiveness in maintaining frame rate and detecting out-of-distribution inputs due to the false positives of a preceding YOLO-based traffic sign detector. Voltage stability assessment is essential for maintaining reliable power grid operations. Stability assessment approaches using deep learning address the shortfalls of the traditional time-domain simulation-based approaches caused by increased system complexity. However, deep learning models are shown vulnerable to adversarial examples in the field of computer vision. While this vulnerability has been noticed by the power grid cybersecurity research, the domain-specific analysis on the requirements imposed upon effective attack implementation is still lacking. Although these attack requirements are usually reasonable in computer vision tasks, they can be stringent in the context of power grids. This thesis conducts a systematic investigation on the attack requirements and credibility of six representative adversarial example attacks based on a voltage stability assessment application for the New England 10-machine 39-bus power system. The investigation in this thesis shows that (1) compromising about half the transmission system buses' voltage traces is a rule-of-thumb attack requirement; (2) the universal adversarial perturbations regardless of the original clean voltage trajectory possess the same credibility as the widely studied false data injection attacks on power grid state estimation, while the input-specific adversarial perturbations are less credible; (3) the prevailing strong adversarial training thwarts the universal perturbations but fails in defending certain input-specific perturbations. To advance defense to cope with both universal and input-specific adversarial examples, this thesis proposes a new approach that simultaneously estimates the predictive uncertainty of any given input of voltage trajectory and thwarts the attacks effectively. To summarize, this thesis studies the threat and countermeasures for the adversarial example attack as an ongoing concern for the safety-critical autonomous cyber-physical systems. It develops DeepMTD and Sardino, which are two dynamic ensemble-based defenses designed under the strategy of moving target defense, to effectively counteract the adaptive adversarial example adversary for embedded deep visual sensing. It also conducts the systematic requirement investigation and credibility analysis of adversarial example attack against the power grid voltage stability assessment and develops effective countermeasure. Doctor of Philosophy 2022-08-18T08:07:34Z 2022-08-18T08:07:34Z 2022 Thesis-Doctor of Philosophy Song, Q. (2022). Improving security of autonomous cyber-physical systems against adversarial examples. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/161165 https://hdl.handle.net/10356/161165 10.32657/10356/161165 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering Song, Qun Improving security of autonomous cyber-physical systems against adversarial examples |
| title | Improving security of autonomous cyber-physical systems against adversarial examples |
| title_full | Improving security of autonomous cyber-physical systems against adversarial examples |
| title_fullStr | Improving security of autonomous cyber-physical systems against adversarial examples |
| title_full_unstemmed | Improving security of autonomous cyber-physical systems against adversarial examples |
| title_short | Improving security of autonomous cyber-physical systems against adversarial examples |
| title_sort | improving security of autonomous cyber physical systems against adversarial examples |
| topic | Engineering::Computer science and engineering |
| url | https://hdl.handle.net/10356/161165 |
| work_keys_str_mv | AT songqun improvingsecurityofautonomouscyberphysicalsystemsagainstadversarialexamples |