The graphical lasso: New insights and alternatives
The graphical lasso [5] is an algorithm for learning the structure in an undirected Gaussian graphical model, using ℓ[subscript 1] regularization to control the number of zeros in the precision matrix Θ = Σ[superscript −1] [2, 11]. The R package GLASSO [5] is popular, fast, and allows one to efficie...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Mathematical Statistics
2013
|
| Online Access: | http://hdl.handle.net/1721.1/80364 https://orcid.org/0000-0003-1384-9743 |
| _version_ | 1826210208134725632 |
|---|---|
| author | Mazumder, Rahul Hastie, Trevor |
| author2 | Massachusetts Institute of Technology. Operations Research Center |
| author_facet | Massachusetts Institute of Technology. Operations Research Center Mazumder, Rahul Hastie, Trevor |
| author_sort | Mazumder, Rahul |
| collection | MIT |
| description | The graphical lasso [5] is an algorithm for learning the structure in an undirected Gaussian graphical model, using ℓ[subscript 1] regularization to control the number of zeros in the precision matrix Θ = Σ[superscript −1] [2, 11]. The R package GLASSO [5] is popular, fast, and allows one to efficiently build a path of models for different values of the tuning parameter. Convergence of GLASSO can be tricky; the converged precision matrix might not be the inverse of the estimated covariance, and occasionally it fails to converge with warm starts. In this paper we explain this behavior, and propose new algorithms that appear to outperform GLASSO.
By studying the “normal equations” we see that, GLASSO is solving the dual of the graphical lasso penalized likelihood, by block coordinate ascent; a result which can also be found in [2]. In this dual, the target of estimation is Σ, the covariance matrix, rather than the precision matrix Θ. We propose similar primal algorithms P-GLASSO and DP-GLASSO, that also operate by block-coordinate descent, where Θ is the optimization target. We study all of these algorithms, and in particular different approaches to solving their coordinate sub-problems. We conclude that DP-GLASSO is superior from several points of view. |
| first_indexed | 2024-09-23T14:45:21Z |
| format | Article |
| id | mit-1721.1/80364 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:45:21Z |
| publishDate | 2013 |
| publisher | Institute of Mathematical Statistics |
| record_format | dspace |
| spelling | mit-1721.1/803642022-09-29T10:22:28Z The graphical lasso: New insights and alternatives Mazumder, Rahul Hastie, Trevor Massachusetts Institute of Technology. Operations Research Center Mazumder, Rahul The graphical lasso [5] is an algorithm for learning the structure in an undirected Gaussian graphical model, using ℓ[subscript 1] regularization to control the number of zeros in the precision matrix Θ = Σ[superscript −1] [2, 11]. The R package GLASSO [5] is popular, fast, and allows one to efficiently build a path of models for different values of the tuning parameter. Convergence of GLASSO can be tricky; the converged precision matrix might not be the inverse of the estimated covariance, and occasionally it fails to converge with warm starts. In this paper we explain this behavior, and propose new algorithms that appear to outperform GLASSO. By studying the “normal equations” we see that, GLASSO is solving the dual of the graphical lasso penalized likelihood, by block coordinate ascent; a result which can also be found in [2]. In this dual, the target of estimation is Σ, the covariance matrix, rather than the precision matrix Θ. We propose similar primal algorithms P-GLASSO and DP-GLASSO, that also operate by block-coordinate descent, where Θ is the optimization target. We study all of these algorithms, and in particular different approaches to solving their coordinate sub-problems. We conclude that DP-GLASSO is superior from several points of view. National Science Foundation (U.S.) (Grant DMS-1007719) 2013-09-06T16:03:19Z 2013-09-06T16:03:19Z 2012 2012-08 Article http://purl.org/eprint/type/JournalArticle 1935-7524 http://hdl.handle.net/1721.1/80364 Mazumder, Rahul, and Trevor Hastie. “The graphical lasso: New insights and alternatives.” Electronic Journal of Statistics 6, no. 0 (2012): 2125-2149. http://dx.doi.org/10.1214/12-EJS740. https://orcid.org/0000-0003-1384-9743 en_US http://dx.doi.org/10.1214/12-EJS740 Electronic Journal of Statistics Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Institute of Mathematical Statistics Institute of Mathematical Statistics |
| spellingShingle | Mazumder, Rahul Hastie, Trevor The graphical lasso: New insights and alternatives |
| title | The graphical lasso: New insights and alternatives |
| title_full | The graphical lasso: New insights and alternatives |
| title_fullStr | The graphical lasso: New insights and alternatives |
| title_full_unstemmed | The graphical lasso: New insights and alternatives |
| title_short | The graphical lasso: New insights and alternatives |
| title_sort | graphical lasso new insights and alternatives |
| url | http://hdl.handle.net/1721.1/80364 https://orcid.org/0000-0003-1384-9743 |
| work_keys_str_mv | AT mazumderrahul thegraphicallassonewinsightsandalternatives AT hastietrevor thegraphicallassonewinsightsandalternatives AT mazumderrahul graphicallassonewinsightsandalternatives AT hastietrevor graphicallassonewinsightsandalternatives |