Development of learning algorithms for fully connect layers of convolutional neural network
In the field of transfer learning, pre-trained weights are usually used in the Convolutional Layers to get the features of the data and Fully Connected Layers are trained again for another classification task based on new training data. Inspired by the ideas of transfer learning and kinematic con...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Coursework |
| Language: | English |
| Published: |
Nanyang Technological University
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/143189 |
| _version_ | 1826124527595159552 |
|---|---|
| author | Wang, Yiming |
| author2 | CHEAH Chien Chern |
| author_facet | CHEAH Chien Chern Wang, Yiming |
| author_sort | Wang, Yiming |
| collection | NTU |
| description | In the field of transfer learning, pre-trained weights are usually used in the Convolutional Layers to get the features of the data and Fully Connected Layers are trained again for another classification task based on new training data.
Inspired by the ideas of transfer learning and kinematic control algorithm in the field of robotics, this dissertation proposes a new learning algorithm which can be used to train the Fully Connected Layers in a simple way.
It is observed based on the testing results of Cifar-10 that the algorithm is able to achieve a better accuracy at the start of the training while the accuracy of baseline method, Stochastic Gradient Descent, is lower at the beginning. The accuracy of the algorithm improves and becomes stable in the progress of learning.
Thus, the algorithm has the potential of achieving a better performance in lesser epochs of training as compared with traditional Gradient Descent Method. |
| first_indexed | 2024-10-01T06:22:06Z |
| format | Thesis-Master by Coursework |
| id | ntu-10356/143189 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:22:06Z |
| publishDate | 2020 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1431892023-07-04T16:42:11Z Development of learning algorithms for fully connect layers of convolutional neural network Wang, Yiming CHEAH Chien Chern School of Electrical and Electronic Engineering ECCCheah@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics In the field of transfer learning, pre-trained weights are usually used in the Convolutional Layers to get the features of the data and Fully Connected Layers are trained again for another classification task based on new training data. Inspired by the ideas of transfer learning and kinematic control algorithm in the field of robotics, this dissertation proposes a new learning algorithm which can be used to train the Fully Connected Layers in a simple way. It is observed based on the testing results of Cifar-10 that the algorithm is able to achieve a better accuracy at the start of the training while the accuracy of baseline method, Stochastic Gradient Descent, is lower at the beginning. The accuracy of the algorithm improves and becomes stable in the progress of learning. Thus, the algorithm has the potential of achieving a better performance in lesser epochs of training as compared with traditional Gradient Descent Method. Master of Science (Computer Control and Automation) 2020-08-11T12:01:58Z 2020-08-11T12:01:58Z 2020 Thesis-Master by Coursework https://hdl.handle.net/10356/143189 en application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Wang, Yiming Development of learning algorithms for fully connect layers of convolutional neural network |
| title | Development of learning algorithms for fully connect layers of convolutional neural network |
| title_full | Development of learning algorithms for fully connect layers of convolutional neural network |
| title_fullStr | Development of learning algorithms for fully connect layers of convolutional neural network |
| title_full_unstemmed | Development of learning algorithms for fully connect layers of convolutional neural network |
| title_short | Development of learning algorithms for fully connect layers of convolutional neural network |
| title_sort | development of learning algorithms for fully connect layers of convolutional neural network |
| topic | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics |
| url | https://hdl.handle.net/10356/143189 |
| work_keys_str_mv | AT wangyiming developmentoflearningalgorithmsforfullyconnectlayersofconvolutionalneuralnetwork |