Performance Comparison of CNN Models Using Gradient Flow Analysis
Convolutional neural networks (CNNs) are widely used among the various deep learning techniques available because of their superior performance in the fields of computer vision and natural language processing. CNNs can effectively extract the locality and correlation of input data using structures i...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Informatics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-9709/8/3/53 |
| _version_ | 1797518801723457536 |
|---|---|
| author | Seol-Hyun Noh |
| author_facet | Seol-Hyun Noh |
| author_sort | Seol-Hyun Noh |
| collection | DOAJ |
| description | Convolutional neural networks (CNNs) are widely used among the various deep learning techniques available because of their superior performance in the fields of computer vision and natural language processing. CNNs can effectively extract the locality and correlation of input data using structures in which convolutional layers are successively applied to the input data. In general, the performance of neural networks has improved as the depth of CNNs has increased. However, an increase in the depth of a CNN is not always accompanied by an increase in the accuracy of the neural network. This is because the gradient vanishing problem may arise, causing the weights of the weighted layers to fail to converge. Accordingly, the gradient flows of the VGGNet, ResNet, SENet, and DenseNet models were analyzed and compared in this study, and the reasons for the differences in the error rate performances of the models were derived. |
| first_indexed | 2024-03-10T07:34:37Z |
| format | Article |
| id | doaj.art-394aa58bcf8b424a92d40bf36b9c748a |
| institution | Directory Open Access Journal |
| issn | 2227-9709 |
| language | English |
| last_indexed | 2024-03-10T07:34:37Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Informatics |
| spelling | doaj.art-394aa58bcf8b424a92d40bf36b9c748a2023-11-22T13:34:49ZengMDPI AGInformatics2227-97092021-08-01835310.3390/informatics8030053Performance Comparison of CNN Models Using Gradient Flow AnalysisSeol-Hyun Noh0Department of Statistical Data Science, ICT Convergence Engineering, Anyang University, Anyang 14028, KoreaConvolutional neural networks (CNNs) are widely used among the various deep learning techniques available because of their superior performance in the fields of computer vision and natural language processing. CNNs can effectively extract the locality and correlation of input data using structures in which convolutional layers are successively applied to the input data. In general, the performance of neural networks has improved as the depth of CNNs has increased. However, an increase in the depth of a CNN is not always accompanied by an increase in the accuracy of the neural network. This is because the gradient vanishing problem may arise, causing the weights of the weighted layers to fail to converge. Accordingly, the gradient flows of the VGGNet, ResNet, SENet, and DenseNet models were analyzed and compared in this study, and the reasons for the differences in the error rate performances of the models were derived.https://www.mdpi.com/2227-9709/8/3/53CNNgradient vanishing problemgradient flowperformance comparisonerror rate |
| spellingShingle | Seol-Hyun Noh Performance Comparison of CNN Models Using Gradient Flow Analysis Informatics CNN gradient vanishing problem gradient flow performance comparison error rate |
| title | Performance Comparison of CNN Models Using Gradient Flow Analysis |
| title_full | Performance Comparison of CNN Models Using Gradient Flow Analysis |
| title_fullStr | Performance Comparison of CNN Models Using Gradient Flow Analysis |
| title_full_unstemmed | Performance Comparison of CNN Models Using Gradient Flow Analysis |
| title_short | Performance Comparison of CNN Models Using Gradient Flow Analysis |
| title_sort | performance comparison of cnn models using gradient flow analysis |
| topic | CNN gradient vanishing problem gradient flow performance comparison error rate |
| url | https://www.mdpi.com/2227-9709/8/3/53 |
| work_keys_str_mv | AT seolhyunnoh performancecomparisonofcnnmodelsusinggradientflowanalysis |