NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
Convolutional Neural Networks (CNNs) are very popular in many fields including computer vision, speech recognition, natural language processing, etc. Though deep learning leads to groundbreaking performance in those domains, the networks used are very computationally demanding and are far from being...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/9/17/2144 |
| _version_ | 1797521139538329600 |
|---|---|
| author | Chaim Baskin Evgenii Zheltonozhkii Tal Rozen Natan Liss Yoav Chai Eli Schwartz Raja Giryes Alexander M. Bronstein Avi Mendelson |
| author_facet | Chaim Baskin Evgenii Zheltonozhkii Tal Rozen Natan Liss Yoav Chai Eli Schwartz Raja Giryes Alexander M. Bronstein Avi Mendelson |
| author_sort | Chaim Baskin |
| collection | DOAJ |
| description | Convolutional Neural Networks (CNNs) are very popular in many fields including computer vision, speech recognition, natural language processing, etc. Though deep learning leads to groundbreaking performance in those domains, the networks used are very computationally demanding and are far from being able to perform in real-time applications even on a GPU, which is not power efficient and therefore does not suit low power systems such as mobile devices. To overcome this challenge, some solutions have been proposed for quantizing the weights and activations of these networks, which accelerate the runtime significantly. Yet, this acceleration comes at the cost of a larger error unless spatial adjustments are carried out. The method proposed in this work trains quantized neural networks by noise injection and a learned clamping, which improve accuracy. This leads to state-of-the-art results on various regression and classification tasks, e.g., ImageNet classification with architectures such as ResNet-18/34/50 with as low as 3 bit weights and activations. We implement the proposed solution on an FPGA to demonstrate its applicability for low-power real-time applications. The quantization code will become publicly available upon acceptance. |
| first_indexed | 2024-03-10T08:07:23Z |
| format | Article |
| id | doaj.art-f2f13435868c4051ab84e296a387e2a1 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T08:07:23Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-f2f13435868c4051ab84e296a387e2a12023-11-22T10:58:32ZengMDPI AGMathematics2227-73902021-09-01917214410.3390/math9172144NICE: Noise Injection and Clamping Estimation for Neural Network QuantizationChaim Baskin0Evgenii Zheltonozhkii1Tal Rozen2Natan Liss3Yoav Chai4Eli Schwartz5Raja Giryes6Alexander M. Bronstein7Avi Mendelson8Department of Computer Science, Technion, Haifa 3200003, IsraelDepartment of Computer Science, Technion, Haifa 3200003, IsraelDepartment of Electrical Engineering, Technion, Haifa 3200003, IsraelDepartment of Electrical Engineering, Technion, Haifa 3200003, IsraelSchool of Electrical Engineering, Tel-Aviv University, Tel-Aviv 6997801, IsraelSchool of Electrical Engineering, Tel-Aviv University, Tel-Aviv 6997801, IsraelSchool of Electrical Engineering, Tel-Aviv University, Tel-Aviv 6997801, IsraelDepartment of Computer Science, Technion, Haifa 3200003, IsraelDepartment of Computer Science, Technion, Haifa 3200003, IsraelConvolutional Neural Networks (CNNs) are very popular in many fields including computer vision, speech recognition, natural language processing, etc. Though deep learning leads to groundbreaking performance in those domains, the networks used are very computationally demanding and are far from being able to perform in real-time applications even on a GPU, which is not power efficient and therefore does not suit low power systems such as mobile devices. To overcome this challenge, some solutions have been proposed for quantizing the weights and activations of these networks, which accelerate the runtime significantly. Yet, this acceleration comes at the cost of a larger error unless spatial adjustments are carried out. The method proposed in this work trains quantized neural networks by noise injection and a learned clamping, which improve accuracy. This leads to state-of-the-art results on various regression and classification tasks, e.g., ImageNet classification with architectures such as ResNet-18/34/50 with as low as 3 bit weights and activations. We implement the proposed solution on an FPGA to demonstrate its applicability for low-power real-time applications. The quantization code will become publicly available upon acceptance.https://www.mdpi.com/2227-7390/9/17/2144neural networkslow powerquantizationCNN architecture |
| spellingShingle | Chaim Baskin Evgenii Zheltonozhkii Tal Rozen Natan Liss Yoav Chai Eli Schwartz Raja Giryes Alexander M. Bronstein Avi Mendelson NICE: Noise Injection and Clamping Estimation for Neural Network Quantization Mathematics neural networks low power quantization CNN architecture |
| title | NICE: Noise Injection and Clamping Estimation for Neural Network Quantization |
| title_full | NICE: Noise Injection and Clamping Estimation for Neural Network Quantization |
| title_fullStr | NICE: Noise Injection and Clamping Estimation for Neural Network Quantization |
| title_full_unstemmed | NICE: Noise Injection and Clamping Estimation for Neural Network Quantization |
| title_short | NICE: Noise Injection and Clamping Estimation for Neural Network Quantization |
| title_sort | nice noise injection and clamping estimation for neural network quantization |
| topic | neural networks low power quantization CNN architecture |
| url | https://www.mdpi.com/2227-7390/9/17/2144 |
| work_keys_str_mv | AT chaimbaskin nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT evgeniizheltonozhkii nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT talrozen nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT natanliss nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT yoavchai nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT elischwartz nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT rajagiryes nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT alexandermbronstein nicenoiseinjectionandclampingestimationforneuralnetworkquantization AT avimendelson nicenoiseinjectionandclampingestimationforneuralnetworkquantization |