Learning Latent Tree Graphical Models
We study the problem of learning a latent tree graphical model where samples are available only from a subset of variables. We propose two consistent and computationally efficient algorithms for learning minimal latent trees, that is, trees without any redundant hidden nodes. Unlike many existing me...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
CrossRef test prefix
2012
|
| Online Access: | http://hdl.handle.net/1721.1/71241 https://orcid.org/0000-0003-0149-5888 |
| _version_ | 1826211089056006144 |
|---|---|
| author | Choi, Myung Jin Willsky, Alan S. Tan, Vincent Y. F. Anandkumar, Animashree |
| author2 | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems |
| author_facet | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Choi, Myung Jin Willsky, Alan S. Tan, Vincent Y. F. Anandkumar, Animashree |
| author_sort | Choi, Myung Jin |
| collection | MIT |
| description | We study the problem of learning a latent tree graphical model where samples are available only from a subset of variables. We propose two consistent and computationally efficient algorithms for learning minimal latent trees, that is, trees without any redundant hidden nodes. Unlike many existing methods, the observed nodes (or variables) are not constrained to be leaf nodes. Our algorithms can be applied to both discrete and Gaussian random variables and our learned models are such that all the observed and latent variables have the same domain (state space). Our first algorithm, recursive grouping, builds the latent tree recursively by identifying sibling groups using so-called information distances. One of the main contributions of this work is our second algorithm, which we refer to as CLGrouping. CLGrouping starts with a pre-processing procedure in which a tree over the observed variables is constructed. This global step groups the observed nodes that are likely to be close to each other in the true latent tree, thereby guiding subsequent recursive grouping (or equivalent procedures such as neighbor-joining) on much smaller subsets of variables. This results in more accurate and efficient learning of latent trees. We also present regularized versions of our algorithms that learn latent tree approximations of arbitrary distributions. We compare the proposed algorithms to other methods by performing extensive numerical experiments on various latent tree graphical models such as hidden Markov models and star graphs. In addition, we demonstrate the applicability of our methods on real-world data sets by modeling the dependency structure of monthly stock returns in the S&P index and of the words in the 20 newsgroups data set. |
| first_indexed | 2024-09-23T15:00:23Z |
| format | Article |
| id | mit-1721.1/71241 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:00:23Z |
| publishDate | 2012 |
| publisher | CrossRef test prefix |
| record_format | dspace |
| spelling | mit-1721.1/712412022-09-29T11:59:44Z Learning Latent Tree Graphical Models Choi, Myung Jin Willsky, Alan S. Tan, Vincent Y. F. Anandkumar, Animashree Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Willsky, Alan S. Choi, Myung Jin Willsky, Alan S. We study the problem of learning a latent tree graphical model where samples are available only from a subset of variables. We propose two consistent and computationally efficient algorithms for learning minimal latent trees, that is, trees without any redundant hidden nodes. Unlike many existing methods, the observed nodes (or variables) are not constrained to be leaf nodes. Our algorithms can be applied to both discrete and Gaussian random variables and our learned models are such that all the observed and latent variables have the same domain (state space). Our first algorithm, recursive grouping, builds the latent tree recursively by identifying sibling groups using so-called information distances. One of the main contributions of this work is our second algorithm, which we refer to as CLGrouping. CLGrouping starts with a pre-processing procedure in which a tree over the observed variables is constructed. This global step groups the observed nodes that are likely to be close to each other in the true latent tree, thereby guiding subsequent recursive grouping (or equivalent procedures such as neighbor-joining) on much smaller subsets of variables. This results in more accurate and efficient learning of latent trees. We also present regularized versions of our algorithms that learn latent tree approximations of arbitrary distributions. We compare the proposed algorithms to other methods by performing extensive numerical experiments on various latent tree graphical models such as hidden Markov models and star graphs. In addition, we demonstrate the applicability of our methods on real-world data sets by modeling the dependency structure of monthly stock returns in the S&P index and of the words in the 20 newsgroups data set. 2012-06-28T12:30:26Z 2012-06-28T12:30:26Z 2011-05 2011-02 Article http://purl.org/eprint/type/JournalArticle 1532-4435 1533-7928 http://hdl.handle.net/1721.1/71241 Choi, Myung Jin et al. "Learning Latent Tree Graphical Models" Journal of Machine Learning Research 12 (2011). © JMLR 2011 https://orcid.org/0000-0003-0149-5888 en_US http://jmlr.csail.mit.edu/papers/v12/choi11b.html Journal of Machine Learning Research Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf CrossRef test prefix MIT Press |
| spellingShingle | Choi, Myung Jin Willsky, Alan S. Tan, Vincent Y. F. Anandkumar, Animashree Learning Latent Tree Graphical Models |
| title | Learning Latent Tree Graphical Models |
| title_full | Learning Latent Tree Graphical Models |
| title_fullStr | Learning Latent Tree Graphical Models |
| title_full_unstemmed | Learning Latent Tree Graphical Models |
| title_short | Learning Latent Tree Graphical Models |
| title_sort | learning latent tree graphical models |
| url | http://hdl.handle.net/1721.1/71241 https://orcid.org/0000-0003-0149-5888 |
| work_keys_str_mv | AT choimyungjin learninglatenttreegraphicalmodels AT willskyalans learninglatenttreegraphicalmodels AT tanvincentyf learninglatenttreegraphicalmodels AT anandkumaranimashree learninglatenttreegraphicalmodels |