Efficient Continuous Pareto Exploration in Multi-Task Learning
Tasks in multi-task learning often correlate, conflict, or even compete with each other. As a result, a single solution that is optimal for all tasks rarely exists. Recent papers introduced the concept of Pareto optimality to this field and directly cast multi-task learning as multiobjective optimiz...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/150297 |
| _version_ | 1826204527801401344 |
|---|---|
| author | Ma, Pingchuan |
| author2 | Matusik, Wojciech |
| author_facet | Matusik, Wojciech Ma, Pingchuan |
| author_sort | Ma, Pingchuan |
| collection | MIT |
| description | Tasks in multi-task learning often correlate, conflict, or even compete with each other. As a result, a single solution that is optimal for all tasks rarely exists. Recent papers introduced the concept of Pareto optimality to this field and directly cast multi-task learning as multiobjective optimization problems, but solutions returned by existing methods are typically finite, sparse, and discrete. We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems. We scale up theoretical results in multi-objective optimization to modern machine learning problems by proposing a sample-based sparse linear system, for which standard Hessian-free solvers in machine learning can be applied. We compare our method to the state-of-the-art algorithms and demonstrate its usage of analyzing local Pareto sets on various multi-task classification and regression problems. The experimental results confirm that our algorithm reveals the primary directions in local Pareto sets for trade-off balancing, finds more solutions with different trade-offs efficiently, and scales well to tasks with millions of parameters. |
| first_indexed | 2024-09-23T12:56:56Z |
| format | Thesis |
| id | mit-1721.1/150297 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T12:56:56Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1502972023-04-01T03:31:11Z Efficient Continuous Pareto Exploration in Multi-Task Learning Ma, Pingchuan Matusik, Wojciech Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Tasks in multi-task learning often correlate, conflict, or even compete with each other. As a result, a single solution that is optimal for all tasks rarely exists. Recent papers introduced the concept of Pareto optimality to this field and directly cast multi-task learning as multiobjective optimization problems, but solutions returned by existing methods are typically finite, sparse, and discrete. We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems. We scale up theoretical results in multi-objective optimization to modern machine learning problems by proposing a sample-based sparse linear system, for which standard Hessian-free solvers in machine learning can be applied. We compare our method to the state-of-the-art algorithms and demonstrate its usage of analyzing local Pareto sets on various multi-task classification and regression problems. The experimental results confirm that our algorithm reveals the primary directions in local Pareto sets for trade-off balancing, finds more solutions with different trade-offs efficiently, and scales well to tasks with millions of parameters. S.M. 2023-03-31T14:45:54Z 2023-03-31T14:45:54Z 2023-02 2023-02-28T14:36:08.183Z Thesis https://hdl.handle.net/1721.1/150297 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Ma, Pingchuan Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title | Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title_full | Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title_fullStr | Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title_full_unstemmed | Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title_short | Efficient Continuous Pareto Exploration in Multi-Task Learning |
| title_sort | efficient continuous pareto exploration in multi task learning |
| url | https://hdl.handle.net/1721.1/150297 |
| work_keys_str_mv | AT mapingchuan efficientcontinuousparetoexplorationinmultitasklearning |