Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm
Utilizing pre-trained models involves fully or partially using pre-trained parameters as initialization. In general, configuring a pre-trained model demands practitioners’ knowledge about problems or an exhaustive trial–error experiment according to a given task. In this paper, we propose tuning tra...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/11/19/2985 |
| _version_ | 1797479900484993024 |
|---|---|
| author | Jae-Cheol Jeong Gwang-Hyun Yu Min-Gyu Song Dang Thanh Vu Le Hoang Anh Young-Ae Jung Yoon-A Choi Tai-Won Um Jin-Young Kim |
| author_facet | Jae-Cheol Jeong Gwang-Hyun Yu Min-Gyu Song Dang Thanh Vu Le Hoang Anh Young-Ae Jung Yoon-A Choi Tai-Won Um Jin-Young Kim |
| author_sort | Jae-Cheol Jeong |
| collection | DOAJ |
| description | Utilizing pre-trained models involves fully or partially using pre-trained parameters as initialization. In general, configuring a pre-trained model demands practitioners’ knowledge about problems or an exhaustive trial–error experiment according to a given task. In this paper, we propose tuning trainable layers using a genetic algorithm on a pre-trained model that is fine-tuned on single-channel image datasets for a classification task. The single-channel dataset comprises images from grayscale and preprocessed audio signals transformed into a log-Mel spectrogram. Four deep-learning models used in the experimental evaluation employed the pre-trained model with the ImageNet dataset. The proposed genetic algorithm was applied to find the highest fitness for every generation to determine the selective layer tuning of the pre-trained models. Compared to the conventional fine-tuning method and random layer search, our proposed selective layer search with a genetic algorithm achieves higher accuracy, on average, by 9.7% and 1.88% (MNIST-Fashion), 1.31% and 1.14% (UrbanSound8k), and 2.2% and 0.29% (HospitalAlarmSound), respectively. In addition, our searching method can naturally be applied to various datasets of the same task without prior knowledge about the dataset of interest. |
| first_indexed | 2024-03-09T21:52:26Z |
| format | Article |
| id | doaj.art-068369117dc84cfb86148af151911fc1 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T21:52:26Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-068369117dc84cfb86148af151911fc12023-11-23T20:04:23ZengMDPI AGElectronics2079-92922022-09-011119298510.3390/electronics11192985Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic AlgorithmJae-Cheol Jeong0Gwang-Hyun Yu1Min-Gyu Song2Dang Thanh Vu3Le Hoang Anh4Young-Ae Jung5Yoon-A Choi6Tai-Won Um7Jin-Young Kim8Department of Biomedical Engineering, Chonnam National University Hospital, Gwangju 61469, KoreaDepartment of ICT Convergence System Engineering, Chonnam National University, Gwangju 61186, KoreaDepartment of Biomedical Engineering, Chonnam National University Hospital, Gwangju 61469, KoreaDepartment of ICT Convergence System Engineering, Chonnam National University, Gwangju 61186, KoreaDepartment of ICT Convergence System Engineering, Chonnam National University, Gwangju 61186, KoreaDivision of Information Technology Education, Sunmoon University, Asan 31460, KoreaKorea Electric Power Research Institute (KEPRI), Daejeon 34056, KoreaGraduate School of Data Science, Chonnam National University, Gwangju 61186, KoreaDepartment of ICT Convergence System Engineering, Chonnam National University, Gwangju 61186, KoreaUtilizing pre-trained models involves fully or partially using pre-trained parameters as initialization. In general, configuring a pre-trained model demands practitioners’ knowledge about problems or an exhaustive trial–error experiment according to a given task. In this paper, we propose tuning trainable layers using a genetic algorithm on a pre-trained model that is fine-tuned on single-channel image datasets for a classification task. The single-channel dataset comprises images from grayscale and preprocessed audio signals transformed into a log-Mel spectrogram. Four deep-learning models used in the experimental evaluation employed the pre-trained model with the ImageNet dataset. The proposed genetic algorithm was applied to find the highest fitness for every generation to determine the selective layer tuning of the pre-trained models. Compared to the conventional fine-tuning method and random layer search, our proposed selective layer search with a genetic algorithm achieves higher accuracy, on average, by 9.7% and 1.88% (MNIST-Fashion), 1.31% and 1.14% (UrbanSound8k), and 2.2% and 0.29% (HospitalAlarmSound), respectively. In addition, our searching method can naturally be applied to various datasets of the same task without prior knowledge about the dataset of interest.https://www.mdpi.com/2079-9292/11/19/2985deep learningselective layer tuninggenetic algorithmpre-trained model |
| spellingShingle | Jae-Cheol Jeong Gwang-Hyun Yu Min-Gyu Song Dang Thanh Vu Le Hoang Anh Young-Ae Jung Yoon-A Choi Tai-Won Um Jin-Young Kim Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm Electronics deep learning selective layer tuning genetic algorithm pre-trained model |
| title | Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm |
| title_full | Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm |
| title_fullStr | Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm |
| title_full_unstemmed | Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm |
| title_short | Selective Layer Tuning and Performance Study of Pre-Trained Models Using Genetic Algorithm |
| title_sort | selective layer tuning and performance study of pre trained models using genetic algorithm |
| topic | deep learning selective layer tuning genetic algorithm pre-trained model |
| url | https://www.mdpi.com/2079-9292/11/19/2985 |
| work_keys_str_mv | AT jaecheoljeong selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT gwanghyunyu selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT mingyusong selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT dangthanhvu selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT lehoanganh selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT youngaejung selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT yoonachoi selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT taiwonum selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm AT jinyoungkim selectivelayertuningandperformancestudyofpretrainedmodelsusinggeneticalgorithm |