Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification
In the past decade, Convolutional Neural Networks (CNNs) have achieved tremendous success in solving complex classification problems. CNN architectures require an excessive number of computations to achieve high accuracy. However, these models are deficient due to the heavy cost of storage and energ...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9989372/ |
| _version_ | 1797959618520940544 |
|---|---|
| author | Zahra Waheed Shehzad Khalid Syed Mursleen Riaz Sajid Gul Khawaja Rimsha Tariq |
| author_facet | Zahra Waheed Shehzad Khalid Syed Mursleen Riaz Sajid Gul Khawaja Rimsha Tariq |
| author_sort | Zahra Waheed |
| collection | DOAJ |
| description | In the past decade, Convolutional Neural Networks (CNNs) have achieved tremendous success in solving complex classification problems. CNN architectures require an excessive number of computations to achieve high accuracy. However, these models are deficient due to the heavy cost of storage and energy, which prohibits the application of CNNs to resource-constrained edge-devices. Hence, developing aggressive optimization schemes for efficient deployment of CNNs on edge devices has become the most important requirement. To find the optimal approach, we present a resource-limited environment based memory-efficient network compression model for image-level object classification. The main aim is to compress CNN architecture by achieving low computational cost and memory requirements without dropping system’s accuracy. To achieve the said goal, we propose a network compression strategy, that works in a collaborative manner, where Soft Filter Pruning is first applied to reduce the computational cost of the model. In the next step, the model is divided into No-Pruning Layers (NP-Layers) and Pruning Layers (P-Layers). Incremental Quantization is applied to P-Layers due to irregular weights distribution, while for NP-Layers, we propose a novel Optimized Quantization algorithm for the quantization of weights up to optimal levels obtained from the Optimizer. This scheme is designed to achieve the best trade-off between compression ratio and accuracy of the model. Our proposed system is validated for image-level object classification on LeNet-5, CIFAR-quick, and VGG-16 networks using MNIST, CIFAR-10, and ImageNet ILSVRC2012 datasets respectively. We have achieved high compression ratio with negligible accuracy drop, outperforming the state-of-the-art methods. |
| first_indexed | 2024-04-11T00:35:18Z |
| format | Article |
| id | doaj.art-d23a448e391f448087bff72e78a5ecdf |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T00:35:18Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d23a448e391f448087bff72e78a5ecdf2023-01-07T00:00:52ZengIEEEIEEE Access2169-35362023-01-01111386140610.1109/ACCESS.2022.32300089989372Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object ClassificationZahra Waheed0https://orcid.org/0000-0001-9160-5774Shehzad Khalid1https://orcid.org/0000-0003-0899-7354Syed Mursleen Riaz2https://orcid.org/0000-0003-4555-1693Sajid Gul Khawaja3Rimsha Tariq4Department of Computer Engineering, Bahria University, Islamabad, PakistanDepartment of Computer Engineering, Bahria University, Islamabad, PakistanDepartment of Computer Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology, Rawalpindi, PakistanDepartment of Computer Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology, Rawalpindi, PakistanDepartment of Computer Engineering, College of Electrical and Mechanical Engineering, National University of Sciences and Technology, Rawalpindi, PakistanIn the past decade, Convolutional Neural Networks (CNNs) have achieved tremendous success in solving complex classification problems. CNN architectures require an excessive number of computations to achieve high accuracy. However, these models are deficient due to the heavy cost of storage and energy, which prohibits the application of CNNs to resource-constrained edge-devices. Hence, developing aggressive optimization schemes for efficient deployment of CNNs on edge devices has become the most important requirement. To find the optimal approach, we present a resource-limited environment based memory-efficient network compression model for image-level object classification. The main aim is to compress CNN architecture by achieving low computational cost and memory requirements without dropping system’s accuracy. To achieve the said goal, we propose a network compression strategy, that works in a collaborative manner, where Soft Filter Pruning is first applied to reduce the computational cost of the model. In the next step, the model is divided into No-Pruning Layers (NP-Layers) and Pruning Layers (P-Layers). Incremental Quantization is applied to P-Layers due to irregular weights distribution, while for NP-Layers, we propose a novel Optimized Quantization algorithm for the quantization of weights up to optimal levels obtained from the Optimizer. This scheme is designed to achieve the best trade-off between compression ratio and accuracy of the model. Our proposed system is validated for image-level object classification on LeNet-5, CIFAR-quick, and VGG-16 networks using MNIST, CIFAR-10, and ImageNet ILSVRC2012 datasets respectively. We have achieved high compression ratio with negligible accuracy drop, outperforming the state-of-the-art methods.https://ieeexplore.ieee.org/document/9989372/Memory-efficient network compressionpruningquantizationimage-level object classificationresource-restricted edge-devices |
| spellingShingle | Zahra Waheed Shehzad Khalid Syed Mursleen Riaz Sajid Gul Khawaja Rimsha Tariq Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification IEEE Access Memory-efficient network compression pruning quantization image-level object classification resource-restricted edge-devices |
| title | Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification |
| title_full | Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification |
| title_fullStr | Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification |
| title_full_unstemmed | Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification |
| title_short | Resource-Restricted Environments Based Memory-Efficient Compressed Convolutional Neural Network Model for Image-Level Object Classification |
| title_sort | resource restricted environments based memory efficient compressed convolutional neural network model for image level object classification |
| topic | Memory-efficient network compression pruning quantization image-level object classification resource-restricted edge-devices |
| url | https://ieeexplore.ieee.org/document/9989372/ |
| work_keys_str_mv | AT zahrawaheed resourcerestrictedenvironmentsbasedmemoryefficientcompressedconvolutionalneuralnetworkmodelforimagelevelobjectclassification AT shehzadkhalid resourcerestrictedenvironmentsbasedmemoryefficientcompressedconvolutionalneuralnetworkmodelforimagelevelobjectclassification AT syedmursleenriaz resourcerestrictedenvironmentsbasedmemoryefficientcompressedconvolutionalneuralnetworkmodelforimagelevelobjectclassification AT sajidgulkhawaja resourcerestrictedenvironmentsbasedmemoryefficientcompressedconvolutionalneuralnetworkmodelforimagelevelobjectclassification AT rimshatariq resourcerestrictedenvironmentsbasedmemoryefficientcompressedconvolutionalneuralnetworkmodelforimagelevelobjectclassification |