Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices
Deep learning has celebrated resounding successes in many application areas of relevance to the Internet of Things (IoT), such as computer vision and machine listening. These technologies must ultimately be brought directly to the edge to fully harness the power of deep leaning for the IoT. The obvi...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9672158/ |
| _version_ | 1798014745480003584 |
|---|---|
| author | Md Mohaimenuzzaman Christoph Bergmeir Bernd Meyer |
| author_facet | Md Mohaimenuzzaman Christoph Bergmeir Bernd Meyer |
| author_sort | Md Mohaimenuzzaman |
| collection | DOAJ |
| description | Deep learning has celebrated resounding successes in many application areas of relevance to the Internet of Things (IoT), such as computer vision and machine listening. These technologies must ultimately be brought directly to the edge to fully harness the power of deep leaning for the IoT. The obvious challenge is that deep learning techniques can only be implemented on strictly resource-constrained edge devices if the models are radically downsized. This task relies on different model compression techniques, such as network pruning, quantization, and the recent advancement of XNOR-Net. This study examines the suitability of these techniques for audio classification on microcontrollers. We present an application of XNOR-Net for end-to-end raw audio classification and a comprehensive empirical study comparing this approach with pruning-and-quantization methods. We show that raw audio classification with XNOR yields comparable performance to regular full precision networks for small numbers of classes while reducing memory requirements 32-fold and computation requirements 58-fold. However, as the number of classes increases significantly, performance degrades, and pruning-and-quantization based compression techniques take over as the preferred technique being able to satisfy the same space constraints but requiring approximately 8x more computation. We show that these insights are consistent between raw audio classification and image classification using standard benchmark sets. To the best of our knowledge, this is the first study to apply XNOR to end-to-end audio classification and evaluate it in the context of alternative techniques. All codes are publicly available on GitHub. |
| first_indexed | 2024-04-11T15:23:13Z |
| format | Article |
| id | doaj.art-1b8a9458eced4c28a627e659bac55416 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T15:23:13Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1b8a9458eced4c28a627e659bac554162022-12-22T04:16:19ZengIEEEIEEE Access2169-35362022-01-01106696670710.1109/ACCESS.2022.31408079672158Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-DevicesMd Mohaimenuzzaman0https://orcid.org/0000-0002-9798-8136Christoph Bergmeir1Bernd Meyer2Department of Data Science and AI, Monash University, Clayton, VIC, AustraliaDepartment of Data Science and AI, Monash University, Clayton, VIC, AustraliaDepartment of Data Science and AI, Monash University, Clayton, VIC, AustraliaDeep learning has celebrated resounding successes in many application areas of relevance to the Internet of Things (IoT), such as computer vision and machine listening. These technologies must ultimately be brought directly to the edge to fully harness the power of deep leaning for the IoT. The obvious challenge is that deep learning techniques can only be implemented on strictly resource-constrained edge devices if the models are radically downsized. This task relies on different model compression techniques, such as network pruning, quantization, and the recent advancement of XNOR-Net. This study examines the suitability of these techniques for audio classification on microcontrollers. We present an application of XNOR-Net for end-to-end raw audio classification and a comprehensive empirical study comparing this approach with pruning-and-quantization methods. We show that raw audio classification with XNOR yields comparable performance to regular full precision networks for small numbers of classes while reducing memory requirements 32-fold and computation requirements 58-fold. However, as the number of classes increases significantly, performance degrades, and pruning-and-quantization based compression techniques take over as the preferred technique being able to satisfy the same space constraints but requiring approximately 8x more computation. We show that these insights are consistent between raw audio classification and image classification using standard benchmark sets. To the best of our knowledge, this is the first study to apply XNOR to end-to-end audio classification and evaluate it in the context of alternative techniques. All codes are publicly available on GitHub.https://ieeexplore.ieee.org/document/9672158/Sound classificationaudio classificationdeep learningmodel compressionfilter pruningchannel pruning |
| spellingShingle | Md Mohaimenuzzaman Christoph Bergmeir Bernd Meyer Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices IEEE Access Sound classification audio classification deep learning model compression filter pruning channel pruning |
| title | Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices |
| title_full | Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices |
| title_fullStr | Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices |
| title_full_unstemmed | Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices |
| title_short | Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices |
| title_sort | pruning vs xnor net a comprehensive study of deep learning for audio classification on edge devices |
| topic | Sound classification audio classification deep learning model compression filter pruning channel pruning |
| url | https://ieeexplore.ieee.org/document/9672158/ |
| work_keys_str_mv | AT mdmohaimenuzzaman pruningvsxnornetacomprehensivestudyofdeeplearningforaudioclassificationonedgedevices AT christophbergmeir pruningvsxnornetacomprehensivestudyofdeeplearningforaudioclassificationonedgedevices AT berndmeyer pruningvsxnornetacomprehensivestudyofdeeplearningforaudioclassificationonedgedevices |