Optimization and acceleration of convolutional neural networks: A survey
Convolutional neural networks (CNN) is a specialized case of artificial neural networks(ANN) and finds its application in computer vision and parallel distributed computing for processing of massive amount of data generated by sensors and to meet the power constraints of IOT devices. Recent advancem...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-07-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157820304845 |
| _version_ | 1818242114210758656 |
|---|---|
| author | Gousia Habib Shaima Qureshi |
| author_facet | Gousia Habib Shaima Qureshi |
| author_sort | Gousia Habib |
| collection | DOAJ |
| description | Convolutional neural networks (CNN) is a specialized case of artificial neural networks(ANN) and finds its application in computer vision and parallel distributed computing for processing of massive amount of data generated by sensors and to meet the power constraints of IOT devices. Recent advancements in parameter optimization, regularization techniques, improvement in activation functions, corresponding loss functions, advancements in the coted the research of Convolutonal Neural Network’s(CNN’s) in past few years. Training of neural networks is cumbersome and takes a lot of time can take days or even weeks. This limits the application of Convolutional Neural Network(CNN) in real time research fields where computational speed is of utmost importance. Thus there is a need for appropriate and enhanced computational speed to meet the requirements of these real time applications.This paper describes CNN in detail summarizes architectural evolution of CNN from 1998 to 2019. Three types of strategies have been explained to enhance the computational speed of CNN at algorithmic level and implementation level. This paper gives detailed insight about computation speed acceleration using Stochastic Gradient Decent(SGD) optimization, Fast convolution and exploiting parallelism challenges in CNN posed by these techniques and recent advancements.The paper also includes detailed view of different framework usage while implementing fast convolution or parallelism techniques. The ultimate aim of the paper to explore all such recent techniques by which we can accelerate the training speed of the CNN’s without compromising the accuracy. |
| first_indexed | 2024-12-12T13:40:05Z |
| format | Article |
| id | doaj.art-52482f9a56fb4397bc4a573e2b4f6f3c |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-12-12T13:40:05Z |
| publishDate | 2022-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-52482f9a56fb4397bc4a573e2b4f6f3c2022-12-22T00:22:49ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782022-07-0134742444268Optimization and acceleration of convolutional neural networks: A surveyGousia Habib0Shaima Qureshi1Corresponding author.; Department of Computer Science and Engineering, National institute of Technology Srinagar, IndiaDepartment of Computer Science and Engineering, National institute of Technology Srinagar, IndiaConvolutional neural networks (CNN) is a specialized case of artificial neural networks(ANN) and finds its application in computer vision and parallel distributed computing for processing of massive amount of data generated by sensors and to meet the power constraints of IOT devices. Recent advancements in parameter optimization, regularization techniques, improvement in activation functions, corresponding loss functions, advancements in the coted the research of Convolutonal Neural Network’s(CNN’s) in past few years. Training of neural networks is cumbersome and takes a lot of time can take days or even weeks. This limits the application of Convolutional Neural Network(CNN) in real time research fields where computational speed is of utmost importance. Thus there is a need for appropriate and enhanced computational speed to meet the requirements of these real time applications.This paper describes CNN in detail summarizes architectural evolution of CNN from 1998 to 2019. Three types of strategies have been explained to enhance the computational speed of CNN at algorithmic level and implementation level. This paper gives detailed insight about computation speed acceleration using Stochastic Gradient Decent(SGD) optimization, Fast convolution and exploiting parallelism challenges in CNN posed by these techniques and recent advancements.The paper also includes detailed view of different framework usage while implementing fast convolution or parallelism techniques. The ultimate aim of the paper to explore all such recent techniques by which we can accelerate the training speed of the CNN’s without compromising the accuracy.http://www.sciencedirect.com/science/article/pii/S1319157820304845SGDReLuHyParADAMNADAM |
| spellingShingle | Gousia Habib Shaima Qureshi Optimization and acceleration of convolutional neural networks: A survey Journal of King Saud University: Computer and Information Sciences SGD ReLu HyPar ADAM NADAM |
| title | Optimization and acceleration of convolutional neural networks: A survey |
| title_full | Optimization and acceleration of convolutional neural networks: A survey |
| title_fullStr | Optimization and acceleration of convolutional neural networks: A survey |
| title_full_unstemmed | Optimization and acceleration of convolutional neural networks: A survey |
| title_short | Optimization and acceleration of convolutional neural networks: A survey |
| title_sort | optimization and acceleration of convolutional neural networks a survey |
| topic | SGD ReLu HyPar ADAM NADAM |
| url | http://www.sciencedirect.com/science/article/pii/S1319157820304845 |
| work_keys_str_mv | AT gousiahabib optimizationandaccelerationofconvolutionalneuralnetworksasurvey AT shaimaqureshi optimizationandaccelerationofconvolutionalneuralnetworksasurvey |