Layer sequence extraction of optimized DNNs using side-channel information leaks
Deep Neural Network (DNN) Intellectual Property (IP) models must be kept undisclosed to avoid revealing trade secrets. Recent works have devised machine learning techniques that leverage on side-channel information leakage of the target platform to reverse engineer DNN architectures. However, these...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/178546 |
| _version_ | 1811679912329740288 |
|---|---|
| author | Sun, Yidan Jiang, Guiyuan Liu, Xinwang He, Peilan Lam, Siew-Kei |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Sun, Yidan Jiang, Guiyuan Liu, Xinwang He, Peilan Lam, Siew-Kei |
| author_sort | Sun, Yidan |
| collection | NTU |
| description | Deep Neural Network (DNN) Intellectual Property (IP) models must be kept undisclosed to avoid revealing trade secrets. Recent works have devised machine learning techniques that leverage on side-channel information leakage of the target platform to reverse engineer DNN architectures. However, these works fail to perform successful attacks on DNNs that have undergone performance optimizations (i.e., operator fusion) using DNN compilers, e.g., Apache Tensor Virtual Machine (TVM). We propose a two-phase attack framework to infer the layer sequences of optimized DNNs through side-channel information leakage. In the first phase, we use a recurrent network with multi-head attention components to learn the intra and interlayer fusion patterns from GPU traces of TVM-optimized DNNs, in order to accurately predict the operation distribution. The second phase uses a model to learn the run-time temporal correlations between operations and layers, which enables the prediction of layer sequence. An encoding strategy is proposed to overcome the convergence issues faced by existing learning-based methods when inferring the layer sequences of optimized DNNs. Extensive experiments show that our learning-based framework outperforms state-of-the-art DNN model extraction techniques. Our framework is also the first to effectively reverse engineer both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) using side-channel leakage. |
| first_indexed | 2024-10-01T03:16:41Z |
| format | Journal Article |
| id | ntu-10356/178546 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:16:41Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1785462024-06-26T00:50:27Z Layer sequence extraction of optimized DNNs using side-channel information leaks Sun, Yidan Jiang, Guiyuan Liu, Xinwang He, Peilan Lam, Siew-Kei School of Computer Science and Engineering College of Computing and Data Science Cyber Security Research Centre @ NTU (CYSREN) Computer and Information Science Side-channel attack Deep neural network Model stealing Deep Neural Network (DNN) Intellectual Property (IP) models must be kept undisclosed to avoid revealing trade secrets. Recent works have devised machine learning techniques that leverage on side-channel information leakage of the target platform to reverse engineer DNN architectures. However, these works fail to perform successful attacks on DNNs that have undergone performance optimizations (i.e., operator fusion) using DNN compilers, e.g., Apache Tensor Virtual Machine (TVM). We propose a two-phase attack framework to infer the layer sequences of optimized DNNs through side-channel information leakage. In the first phase, we use a recurrent network with multi-head attention components to learn the intra and interlayer fusion patterns from GPU traces of TVM-optimized DNNs, in order to accurately predict the operation distribution. The second phase uses a model to learn the run-time temporal correlations between operations and layers, which enables the prediction of layer sequence. An encoding strategy is proposed to overcome the convergence issues faced by existing learning-based methods when inferring the layer sequences of optimized DNNs. Extensive experiments show that our learning-based framework outperforms state-of-the-art DNN model extraction techniques. Our framework is also the first to effectively reverse engineer both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) using side-channel leakage. Ministry of Education (MOE) Nanyang Technological University Submitted/Accepted version This work was supported in part by NTU-DESAY SV Research Program under Grant 2018-0980; and in part by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2, under Grant MOE-T2EP20121-0008. 2024-06-26T00:50:27Z 2024-06-26T00:50:27Z 2024 Journal Article Sun, Y., Jiang, G., Liu, X., He, P. & Lam, S. (2024). Layer sequence extraction of optimized DNNs using side-channel information leaks. IEEE Transactions On Computer-Aided Design of Integrated Circuits and Systems. https://dx.doi.org/10.1109/TCAD.2024.3389554 0278-0070 https://hdl.handle.net/10356/178546 10.1109/TCAD.2024.3389554 en NTU-DESAY SV 2018-0980 MOE-T2EP20121-0008 IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/TCAD.2024.3389554. application/pdf |
| spellingShingle | Computer and Information Science Side-channel attack Deep neural network Model stealing Sun, Yidan Jiang, Guiyuan Liu, Xinwang He, Peilan Lam, Siew-Kei Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title | Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title_full | Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title_fullStr | Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title_full_unstemmed | Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title_short | Layer sequence extraction of optimized DNNs using side-channel information leaks |
| title_sort | layer sequence extraction of optimized dnns using side channel information leaks |
| topic | Computer and Information Science Side-channel attack Deep neural network Model stealing |
| url | https://hdl.handle.net/10356/178546 |
| work_keys_str_mv | AT sunyidan layersequenceextractionofoptimizeddnnsusingsidechannelinformationleaks AT jiangguiyuan layersequenceextractionofoptimizeddnnsusingsidechannelinformationleaks AT liuxinwang layersequenceextractionofoptimizeddnnsusingsidechannelinformationleaks AT hepeilan layersequenceextractionofoptimizeddnnsusingsidechannelinformationleaks AT lamsiewkei layersequenceextractionofoptimizeddnnsusingsidechannelinformationleaks |