A Projected Subgradient Method for Scalable Multi-Task Learning
Recent approaches to multi-task learning have investigated the use of a variety of matrix norm regularization schemes for promoting feature sharing across tasks.In essence, these approaches aim at extending the l1 framework for sparse single task approximation to the multi-task setting. In this pape...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Published: |
2008
|
| Online Access: | http://hdl.handle.net/1721.1/41888 |
| _version_ | 1811072499209732096 |
|---|---|
| author | Quattoni, Ariadna Carreras, Xavier Collins, Michael Darrell, Trevor |
| author2 | Trevor Darrell |
| author_facet | Trevor Darrell Quattoni, Ariadna Carreras, Xavier Collins, Michael Darrell, Trevor |
| author_sort | Quattoni, Ariadna |
| collection | MIT |
| description | Recent approaches to multi-task learning have investigated the use of a variety of matrix norm regularization schemes for promoting feature sharing across tasks.In essence, these approaches aim at extending the l1 framework for sparse single task approximation to the multi-task setting. In this paper we focus on the computational complexity of training a jointly regularized model and propose an optimization algorithm whose complexity is linear with the number of training examples and O(n log n) with n being the number of parameters of the joint model. Our algorithm is based on setting jointly regularized loss minimization as a convex constrained optimization problem for which we develop an efficient projected gradient algorithm. The main contribution of this paper is the derivation of a gradient projection method with l1ââ constraints that can be performed efficiently and which has convergence rates. |
| first_indexed | 2024-09-23T09:06:56Z |
| id | mit-1721.1/41888 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T09:06:56Z |
| publishDate | 2008 |
| record_format | dspace |
| spelling | mit-1721.1/418882019-04-10T17:17:29Z A Projected Subgradient Method for Scalable Multi-Task Learning Quattoni, Ariadna Carreras, Xavier Collins, Michael Darrell, Trevor Trevor Darrell Vision Recent approaches to multi-task learning have investigated the use of a variety of matrix norm regularization schemes for promoting feature sharing across tasks.In essence, these approaches aim at extending the l1 framework for sparse single task approximation to the multi-task setting. In this paper we focus on the computational complexity of training a jointly regularized model and propose an optimization algorithm whose complexity is linear with the number of training examples and O(n log n) with n being the number of parameters of the joint model. Our algorithm is based on setting jointly regularized loss minimization as a convex constrained optimization problem for which we develop an efficient projected gradient algorithm. The main contribution of this paper is the derivation of a gradient projection method with l1ââ constraints that can be performed efficiently and which has convergence rates. 2008-07-24T20:00:14Z 2008-07-24T20:00:14Z 2008-07-23 MIT-CSAIL-TR-2008-045 http://hdl.handle.net/1721.1/41888 Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory 8 p. application/pdf application/postscript |
| spellingShingle | Quattoni, Ariadna Carreras, Xavier Collins, Michael Darrell, Trevor A Projected Subgradient Method for Scalable Multi-Task Learning |
| title | A Projected Subgradient Method for Scalable Multi-Task Learning |
| title_full | A Projected Subgradient Method for Scalable Multi-Task Learning |
| title_fullStr | A Projected Subgradient Method for Scalable Multi-Task Learning |
| title_full_unstemmed | A Projected Subgradient Method for Scalable Multi-Task Learning |
| title_short | A Projected Subgradient Method for Scalable Multi-Task Learning |
| title_sort | projected subgradient method for scalable multi task learning |
| url | http://hdl.handle.net/1721.1/41888 |
| work_keys_str_mv | AT quattoniariadna aprojectedsubgradientmethodforscalablemultitasklearning AT carrerasxavier aprojectedsubgradientmethodforscalablemultitasklearning AT collinsmichael aprojectedsubgradientmethodforscalablemultitasklearning AT darrelltrevor aprojectedsubgradientmethodforscalablemultitasklearning AT quattoniariadna projectedsubgradientmethodforscalablemultitasklearning AT carrerasxavier projectedsubgradientmethodforscalablemultitasklearning AT collinsmichael projectedsubgradientmethodforscalablemultitasklearning AT darrelltrevor projectedsubgradientmethodforscalablemultitasklearning |