Adversarial patch detection

Digital twinning, a fundamental method used in the Metaverse, allows for the virtualization of people, real-world landscapes, and objects. Using machine learning algorithms to process large amounts of data, digital twins can simulate and make decisions based on users’ actions in the physical world....

Full description

Bibliographic Details
Main Author: Yeong, Joash Ler Yuen
Other Authors: Jun Zhao
Format: Final Year Project (FYP)
Language:English
Published: Nanyang Technological University 2022
Subjects:
Online Access:https://hdl.handle.net/10356/162907
Description
Summary:Digital twinning, a fundamental method used in the Metaverse, allows for the virtualization of people, real-world landscapes, and objects. Using machine learning algorithms to process large amounts of data, digital twins can simulate and make decisions based on users’ actions in the physical world. However, the security of these technologies may be jeopardised in the face of adversarial attacks. By introducing adversarial patches that distort perceived data, deep learning models can produce inaccurate predictions. Hence, we focused on a setting where users on the Internet of Vehicles (IoV) are capturing views of the virtual world in real time and identifying these adversarial patches. Unfortunately, the lack of strong computational capacity makes it impractical for IoV sensors to run adversarial patch detection. In this paper, we came up with an edge orchestrator by using deep reinforcement learning to offload the task of detecting adversarial patches to systems that are good at computing while easing the trade-off between accuracy and latency. Experiments were done to show that our proposed system and algorithms work well and are efficient.