Neural Network Compression via Low Frequency Preference
Network pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/12/3144 |
| _version_ | 1797592764793225216 |
|---|---|
| author | Chaoyan Zhang Cheng Li Baolong Guo Nannan Liao |
| author_facet | Chaoyan Zhang Cheng Li Baolong Guo Nannan Liao |
| author_sort | Chaoyan Zhang |
| collection | DOAJ |
| description | Network pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-consider the model characteristics and propose a novel filter pruning method that corresponds to the human visual system, termed Low Frequency Preference (LFP), in the frequency domain. It is essentially an indicator that determines the importance of a filter based on the relative low-frequency components across channels, which can be intuitively understood as a measurement of the “low-frequency components”. When the feature map of a filter has more low-frequency components than the other feature maps, it is considered more crucial and should be preserved during the pruning process. We conduct the proposed LFP on three different scales of datasets through several models and achieve superior performances. The experimental results obtained on the CIFAR datasets and ImageNet dataset demonstrate that our method significantly reduces the model size and FLOPs. The results on the UC Merced dataset show that our approach is also significant for remote sensing image classification. |
| first_indexed | 2024-03-11T01:57:50Z |
| format | Article |
| id | doaj.art-400a6ce342ff462d919da3bf7c78f712 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T01:57:50Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-400a6ce342ff462d919da3bf7c78f7122023-11-18T12:27:05ZengMDPI AGRemote Sensing2072-42922023-06-011512314410.3390/rs15123144Neural Network Compression via Low Frequency PreferenceChaoyan Zhang0Cheng Li1Baolong Guo2Nannan Liao3Institute of Intelligent Control and Image Engineering, Xidian University, Xi’an 710071, ChinaInstitute of Intelligent Control and Image Engineering, Xidian University, Xi’an 710071, ChinaInstitute of Intelligent Control and Image Engineering, Xidian University, Xi’an 710071, ChinaInstitute of Intelligent Control and Image Engineering, Xidian University, Xi’an 710071, ChinaNetwork pruning has been widely used in model compression techniques, and offers a promising prospect for deploying models on devices with limited resources. Nevertheless, existing pruning methods merely consider the importance of feature maps and filters in the spatial domain. In this paper, we re-consider the model characteristics and propose a novel filter pruning method that corresponds to the human visual system, termed Low Frequency Preference (LFP), in the frequency domain. It is essentially an indicator that determines the importance of a filter based on the relative low-frequency components across channels, which can be intuitively understood as a measurement of the “low-frequency components”. When the feature map of a filter has more low-frequency components than the other feature maps, it is considered more crucial and should be preserved during the pruning process. We conduct the proposed LFP on three different scales of datasets through several models and achieve superior performances. The experimental results obtained on the CIFAR datasets and ImageNet dataset demonstrate that our method significantly reduces the model size and FLOPs. The results on the UC Merced dataset show that our approach is also significant for remote sensing image classification.https://www.mdpi.com/2072-4292/15/12/3144model compressionneural network pruningfrequency domainlightweight deep neural networksremote sensing image classification |
| spellingShingle | Chaoyan Zhang Cheng Li Baolong Guo Nannan Liao Neural Network Compression via Low Frequency Preference Remote Sensing model compression neural network pruning frequency domain lightweight deep neural networks remote sensing image classification |
| title | Neural Network Compression via Low Frequency Preference |
| title_full | Neural Network Compression via Low Frequency Preference |
| title_fullStr | Neural Network Compression via Low Frequency Preference |
| title_full_unstemmed | Neural Network Compression via Low Frequency Preference |
| title_short | Neural Network Compression via Low Frequency Preference |
| title_sort | neural network compression via low frequency preference |
| topic | model compression neural network pruning frequency domain lightweight deep neural networks remote sensing image classification |
| url | https://www.mdpi.com/2072-4292/15/12/3144 |
| work_keys_str_mv | AT chaoyanzhang neuralnetworkcompressionvialowfrequencypreference AT chengli neuralnetworkcompressionvialowfrequencypreference AT baolongguo neuralnetworkcompressionvialowfrequencypreference AT nannanliao neuralnetworkcompressionvialowfrequencypreference |