Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means
Convolutional Neural Networks (CNNs) are popular models that are widely used in image classification, target recognition, and other fields. Model compression is a common step in transplanting neural networks into embedded devices, and it is often used in the retraining stage. However, it requires a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/11/4298 |
| _version_ | 1797491581013458944 |
|---|---|
| author | Mingjie Wei Yunping Zhao Xiaowen Chen Chen Li Jianzhuang Lu |
| author_facet | Mingjie Wei Yunping Zhao Xiaowen Chen Chen Li Jianzhuang Lu |
| author_sort | Mingjie Wei |
| collection | DOAJ |
| description | Convolutional Neural Networks (CNNs) are popular models that are widely used in image classification, target recognition, and other fields. Model compression is a common step in transplanting neural networks into embedded devices, and it is often used in the retraining stage. However, it requires a high expenditure of time by retraining weight data to atone for the loss of precision. Unlike in prior designs, we propose a novel model compression approach based on <i>Simon</i><i>k</i>-means, which is specifically designed to support a hardware acceleration scheme. First, we propose an extension algorithm named <i>Simon</i><i>k</i>-means based on simple <i>k</i>-means. We use <i>Simon</i><i>k</i>-means to cluster trained weights in convolutional layers and fully connected layers. Second, we reduce the consumption of hardware resources in data movement and storage by using a data storage and index approach. Finally, we provide the hardware implementation of the compressed CNN accelerator. Our evaluations on several classifications show that our design can achieve 5.27× compression and reduce 74.3% of the multiply–accumulate (MAC) operations in AlexNet on the FASHION-MNIST dataset. |
| first_indexed | 2024-03-10T00:51:21Z |
| format | Article |
| id | doaj.art-cfba72e9731641a1a99491229d594fe6 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T00:51:21Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-cfba72e9731641a1a99491229d594fe62023-11-23T14:51:42ZengMDPI AGSensors1424-82202022-06-012211429810.3390/s22114298Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-MeansMingjie Wei0Yunping Zhao1Xiaowen Chen2Chen Li3Jianzhuang Lu4The College of Computer Science, National University of Defence Technology, Changsha 410000, ChinaThe College of Computer Science, National University of Defence Technology, Changsha 410000, ChinaThe College of Computer Science, National University of Defence Technology, Changsha 410000, ChinaThe College of Computer Science, National University of Defence Technology, Changsha 410000, ChinaThe College of Computer Science, National University of Defence Technology, Changsha 410000, ChinaConvolutional Neural Networks (CNNs) are popular models that are widely used in image classification, target recognition, and other fields. Model compression is a common step in transplanting neural networks into embedded devices, and it is often used in the retraining stage. However, it requires a high expenditure of time by retraining weight data to atone for the loss of precision. Unlike in prior designs, we propose a novel model compression approach based on <i>Simon</i><i>k</i>-means, which is specifically designed to support a hardware acceleration scheme. First, we propose an extension algorithm named <i>Simon</i><i>k</i>-means based on simple <i>k</i>-means. We use <i>Simon</i><i>k</i>-means to cluster trained weights in convolutional layers and fully connected layers. Second, we reduce the consumption of hardware resources in data movement and storage by using a data storage and index approach. Finally, we provide the hardware implementation of the compressed CNN accelerator. Our evaluations on several classifications show that our design can achieve 5.27× compression and reduce 74.3% of the multiply–accumulate (MAC) operations in AlexNet on the FASHION-MNIST dataset.https://www.mdpi.com/1424-8220/22/11/4298convolutional neural networksdeep learningk-meansmodel compressionweight quantization |
| spellingShingle | Mingjie Wei Yunping Zhao Xiaowen Chen Chen Li Jianzhuang Lu Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means Sensors convolutional neural networks deep learning k-means model compression weight quantization |
| title | Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means |
| title_full | Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means |
| title_fullStr | Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means |
| title_full_unstemmed | Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means |
| title_short | Towards Convolutional Neural Network Acceleration and Compression Based on <i>Simon</i><i>k</i>-Means |
| title_sort | towards convolutional neural network acceleration and compression based on i simon i i k i means |
| topic | convolutional neural networks deep learning k-means model compression weight quantization |
| url | https://www.mdpi.com/1424-8220/22/11/4298 |
| work_keys_str_mv | AT mingjiewei towardsconvolutionalneuralnetworkaccelerationandcompressionbasedonisimoniikimeans AT yunpingzhao towardsconvolutionalneuralnetworkaccelerationandcompressionbasedonisimoniikimeans AT xiaowenchen towardsconvolutionalneuralnetworkaccelerationandcompressionbasedonisimoniikimeans AT chenli towardsconvolutionalneuralnetworkaccelerationandcompressionbasedonisimoniikimeans AT jianzhuanglu towardsconvolutionalneuralnetworkaccelerationandcompressionbasedonisimoniikimeans |