DSConv: efficient convolution operator
Quantization is a popular way of increasing the speed and lowering the memory usage of Convolution Neural Networks (CNNs). When labelled training data is available, network weights and activations have successfully been quantized down to 1-bit. The same cannot be said about the scenario when labelle...
| Príomhchruthaitheoirí: | , , |
|---|---|
| Formáid: | Conference item |
| Teanga: | English |
| Foilsithe / Cruthaithe: |
Institute of Electrical and Electronics Engineers
2019
|
| _version_ | 1826313336551112704 |
|---|---|
| author | Gennari do Nascimento, M Prisacariu, V Fawcett, R |
| author_facet | Gennari do Nascimento, M Prisacariu, V Fawcett, R |
| author_sort | Gennari do Nascimento, M |
| collection | OXFORD |
| description | Quantization is a popular way of increasing the speed and lowering the memory usage of Convolution Neural Networks (CNNs). When labelled training data is available, network weights and activations have successfully been quantized down to 1-bit. The same cannot be said about the scenario when labelled training data is not available, e.g. when quantizing a pre-trained model, where current approaches show, at best, no loss of accuracy at 8-bit quantizations. We introduce DSConv, a flexible quantized convolution operator that replaces single-precision operations with their far less expensive integer counterparts, while maintaining the probability distributions over both the kernel weights and the outputs. We test our model as a plug-and-play replacement for standard convolution on most popular neural network architectures, ResNet, DenseNet, GoogLeNet, AlexNet and VGG-Net and demonstrate state-of-the-art results, with less than 1% loss of accuracy, without retraining, using only 4-bit quantization. We also show how a distillation-based adaptation stage with unlabelled data can improve results even further. |
| first_indexed | 2024-03-07T06:45:37Z |
| format | Conference item |
| id | oxford-uuid:fac07d32-bae0-4c5e-acac-fbd946ce9c6a |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:11:25Z |
| publishDate | 2019 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | oxford-uuid:fac07d32-bae0-4c5e-acac-fbd946ce9c6a2024-06-20T13:57:10ZDSConv: efficient convolution operatorConference itemhttp://purl.org/coar/resource_type/c_5794uuid:fac07d32-bae0-4c5e-acac-fbd946ce9c6aEnglishSymplectic ElementsInstitute of Electrical and Electronics Engineers2019Gennari do Nascimento, MPrisacariu, VFawcett, RQuantization is a popular way of increasing the speed and lowering the memory usage of Convolution Neural Networks (CNNs). When labelled training data is available, network weights and activations have successfully been quantized down to 1-bit. The same cannot be said about the scenario when labelled training data is not available, e.g. when quantizing a pre-trained model, where current approaches show, at best, no loss of accuracy at 8-bit quantizations. We introduce DSConv, a flexible quantized convolution operator that replaces single-precision operations with their far less expensive integer counterparts, while maintaining the probability distributions over both the kernel weights and the outputs. We test our model as a plug-and-play replacement for standard convolution on most popular neural network architectures, ResNet, DenseNet, GoogLeNet, AlexNet and VGG-Net and demonstrate state-of-the-art results, with less than 1% loss of accuracy, without retraining, using only 4-bit quantization. We also show how a distillation-based adaptation stage with unlabelled data can improve results even further. |
| spellingShingle | Gennari do Nascimento, M Prisacariu, V Fawcett, R DSConv: efficient convolution operator |
| title | DSConv: efficient convolution operator |
| title_full | DSConv: efficient convolution operator |
| title_fullStr | DSConv: efficient convolution operator |
| title_full_unstemmed | DSConv: efficient convolution operator |
| title_short | DSConv: efficient convolution operator |
| title_sort | dsconv efficient convolution operator |
| work_keys_str_mv | AT gennaridonascimentom dsconvefficientconvolutionoperator AT prisacariuv dsconvefficientconvolutionoperator AT fawcettr dsconvefficientconvolutionoperator |