Adaptive Dynamic Search for Multi-Task Learning
Multi-task learning (MTL) is a learning strategy for solving multiple tasks simultaneously while exploiting commonalities and differences between tasks for improved learning efficiency and prediction performance. Despite its potential, there remain several major challenges to be addressed. First of...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/22/11836 |
| _version_ | 1797465908321452032 |
|---|---|
| author | Eunwoo Kim |
| author_facet | Eunwoo Kim |
| author_sort | Eunwoo Kim |
| collection | DOAJ |
| description | Multi-task learning (MTL) is a learning strategy for solving multiple tasks simultaneously while exploiting commonalities and differences between tasks for improved learning efficiency and prediction performance. Despite its potential, there remain several major challenges to be addressed. First of all, the task performance degrades when the number of tasks to solve increases or the tasks are less related. In addition, finding the prediction model for each task is typically laborious and can be suboptimal. This nature of manually designing the architecture further aggravates the problem when it comes to solving multiple tasks under different computational budgets. In this work, we propose a novel MTL approach to address these issues. The proposed method learns to search in a finely modularized base network dynamically and to discover an optimal prediction model for each instance of a task on the fly while taking the computational costs of the discovered models into account. We evaluate our learning framework on a diverse set of MTL scenarios comprising standard benchmark datasets. We achieve significant improvements in performance for all tested cases compared with existing MTL alternatives. |
| first_indexed | 2024-03-09T18:29:10Z |
| format | Article |
| id | doaj.art-42ee78f2f5cf4fcaab4d09bd56efde63 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T18:29:10Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-42ee78f2f5cf4fcaab4d09bd56efde632023-11-24T07:41:37ZengMDPI AGApplied Sciences2076-34172022-11-0112221183610.3390/app122211836Adaptive Dynamic Search for Multi-Task LearningEunwoo Kim0School of Computer Science and Engineering, Chung-Ang University, Seoul 06974, Republic of KoreaMulti-task learning (MTL) is a learning strategy for solving multiple tasks simultaneously while exploiting commonalities and differences between tasks for improved learning efficiency and prediction performance. Despite its potential, there remain several major challenges to be addressed. First of all, the task performance degrades when the number of tasks to solve increases or the tasks are less related. In addition, finding the prediction model for each task is typically laborious and can be suboptimal. This nature of manually designing the architecture further aggravates the problem when it comes to solving multiple tasks under different computational budgets. In this work, we propose a novel MTL approach to address these issues. The proposed method learns to search in a finely modularized base network dynamically and to discover an optimal prediction model for each instance of a task on the fly while taking the computational costs of the discovered models into account. We evaluate our learning framework on a diverse set of MTL scenarios comprising standard benchmark datasets. We achieve significant improvements in performance for all tested cases compared with existing MTL alternatives.https://www.mdpi.com/2076-3417/12/22/11836multi-task learningdynamic model searchcost-adaptive solutionmemory efficiencydestructive interference |
| spellingShingle | Eunwoo Kim Adaptive Dynamic Search for Multi-Task Learning Applied Sciences multi-task learning dynamic model search cost-adaptive solution memory efficiency destructive interference |
| title | Adaptive Dynamic Search for Multi-Task Learning |
| title_full | Adaptive Dynamic Search for Multi-Task Learning |
| title_fullStr | Adaptive Dynamic Search for Multi-Task Learning |
| title_full_unstemmed | Adaptive Dynamic Search for Multi-Task Learning |
| title_short | Adaptive Dynamic Search for Multi-Task Learning |
| title_sort | adaptive dynamic search for multi task learning |
| topic | multi-task learning dynamic model search cost-adaptive solution memory efficiency destructive interference |
| url | https://www.mdpi.com/2076-3417/12/22/11836 |
| work_keys_str_mv | AT eunwookim adaptivedynamicsearchformultitasklearning |