Finding non-uniform quantization schemes using multi-task Gaussian processes
We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the pro...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2020
|
| _version_ | 1811139661773406208 |
|---|---|
| author | Gennari do Nascimento, M Costain, TW Prisacariu, VA |
| author_facet | Gennari do Nascimento, M Costain, TW Prisacariu, VA |
| author_sort | Gennari do Nascimento, M |
| collection | OXFORD |
| description | We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures. |
| first_indexed | 2024-03-07T05:27:15Z |
| format | Conference item |
| id | oxford-uuid:e0fd0ddd-afd2-4782-b482-95f158b4c32f |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:09:38Z |
| publishDate | 2020 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:e0fd0ddd-afd2-4782-b482-95f158b4c32f2024-06-12T11:44:23ZFinding non-uniform quantization schemes using multi-task Gaussian processesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:e0fd0ddd-afd2-4782-b482-95f158b4c32fEnglishSymplectic ElementsSpringer2020Gennari do Nascimento, MCostain, TWPrisacariu, VAWe propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures. |
| spellingShingle | Gennari do Nascimento, M Costain, TW Prisacariu, VA Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title | Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title_full | Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title_fullStr | Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title_full_unstemmed | Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title_short | Finding non-uniform quantization schemes using multi-task Gaussian processes |
| title_sort | finding non uniform quantization schemes using multi task gaussian processes |
| work_keys_str_mv | AT gennaridonascimentom findingnonuniformquantizationschemesusingmultitaskgaussianprocesses AT costaintw findingnonuniformquantizationschemesusingmultitaskgaussianprocesses AT prisacariuva findingnonuniformquantizationschemesusingmultitaskgaussianprocesses |