Fast Model Selection and Hyperparameter Tuning for Generative Models
Generative models have gained significant attention in recent years. They are increasingly used to estimate the underlying structure of high-dimensional data and artificially generate various kinds of data similar to those from the real world. The performance of generative models depends critically...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/26/2/150 |
| _version_ | 1797298296073486336 |
|---|---|
| author | Luming Chen Sujit K. Ghosh |
| author_facet | Luming Chen Sujit K. Ghosh |
| author_sort | Luming Chen |
| collection | DOAJ |
| description | Generative models have gained significant attention in recent years. They are increasingly used to estimate the underlying structure of high-dimensional data and artificially generate various kinds of data similar to those from the real world. The performance of generative models depends critically on a good set of hyperparameters. Yet, finding the right hyperparameter configuration can be an extremely time-consuming task. In this paper, we focus on speeding up the hyperparameter search through adaptive resource allocation, early stopping underperforming candidates quickly and allocating more computational resources to promising ones by comparing their intermediate performance. The hyperparameter search is formulated as a non-stochastic best-arm identification problem where resources like iterations or training time constrained by some predetermined budget are allocated to different hyperparameter configurations. A procedure which uses hypothesis testing coupled with Successive Halving is proposed to make the resource allocation and early stopping decisions and compares the intermediate performance of generative models by their exponentially weighted Maximum Means Discrepancy (MMD). The experimental results show that the proposed method selects hyperparameter configurations that lead to a significant improvement in the model performance compared to Successive Halving for a wide range of budgets across several real-world applications. |
| first_indexed | 2024-03-07T22:33:52Z |
| format | Article |
| id | doaj.art-b1903ee7e4b4416c89bf27a76ced1efa |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-07T22:33:52Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-b1903ee7e4b4416c89bf27a76ced1efa2024-02-23T15:15:43ZengMDPI AGEntropy1099-43002024-02-0126215010.3390/e26020150Fast Model Selection and Hyperparameter Tuning for Generative ModelsLuming Chen0Sujit K. Ghosh1Department of Statistics, North Carolina State University, Raleigh, NC 27695, USADepartment of Statistics, North Carolina State University, Raleigh, NC 27695, USAGenerative models have gained significant attention in recent years. They are increasingly used to estimate the underlying structure of high-dimensional data and artificially generate various kinds of data similar to those from the real world. The performance of generative models depends critically on a good set of hyperparameters. Yet, finding the right hyperparameter configuration can be an extremely time-consuming task. In this paper, we focus on speeding up the hyperparameter search through adaptive resource allocation, early stopping underperforming candidates quickly and allocating more computational resources to promising ones by comparing their intermediate performance. The hyperparameter search is formulated as a non-stochastic best-arm identification problem where resources like iterations or training time constrained by some predetermined budget are allocated to different hyperparameter configurations. A procedure which uses hypothesis testing coupled with Successive Halving is proposed to make the resource allocation and early stopping decisions and compares the intermediate performance of generative models by their exponentially weighted Maximum Means Discrepancy (MMD). The experimental results show that the proposed method selects hyperparameter configurations that lead to a significant improvement in the model performance compared to Successive Halving for a wide range of budgets across several real-world applications.https://www.mdpi.com/1099-4300/26/2/150integral probability metrichypothesis testingMaximum Mean Discrepancymulti-armed banditsgenerative adversarial networks |
| spellingShingle | Luming Chen Sujit K. Ghosh Fast Model Selection and Hyperparameter Tuning for Generative Models Entropy integral probability metric hypothesis testing Maximum Mean Discrepancy multi-armed bandits generative adversarial networks |
| title | Fast Model Selection and Hyperparameter Tuning for Generative Models |
| title_full | Fast Model Selection and Hyperparameter Tuning for Generative Models |
| title_fullStr | Fast Model Selection and Hyperparameter Tuning for Generative Models |
| title_full_unstemmed | Fast Model Selection and Hyperparameter Tuning for Generative Models |
| title_short | Fast Model Selection and Hyperparameter Tuning for Generative Models |
| title_sort | fast model selection and hyperparameter tuning for generative models |
| topic | integral probability metric hypothesis testing Maximum Mean Discrepancy multi-armed bandits generative adversarial networks |
| url | https://www.mdpi.com/1099-4300/26/2/150 |
| work_keys_str_mv | AT lumingchen fastmodelselectionandhyperparametertuningforgenerativemodels AT sujitkghosh fastmodelselectionandhyperparametertuningforgenerativemodels |