Information theoretic properties of Markov Random Fields, and their algorithmic applications
Markov random fields are a popular model for high-dimensional probability distributions. Over the years, many mathematical, statistical and algorithmic problems on them have been studied. Until recently, the only known algorithms for provably learning them relied on exhaustive search, correlation de...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
2018
|
| Online Access: | http://hdl.handle.net/1721.1/116218 https://orcid.org/0000-0001-7047-0495 |
| _version_ | 1811083439830466560 |
|---|---|
| author | Hamilton, Linus Ulysses Koehler, Frederic Moitra, Ankur |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Hamilton, Linus Ulysses Koehler, Frederic Moitra, Ankur |
| author_sort | Hamilton, Linus Ulysses |
| collection | MIT |
| description | Markov random fields are a popular model for high-dimensional probability distributions. Over the years, many mathematical, statistical and algorithmic problems on them have been studied. Until recently, the only known algorithms for provably learning them relied on exhaustive search, correlation decay or various incoherence assumptions. Bresler [1] gave an algorithm for learning general Ising models on bounded degree graphs. His approach was based on a structural result about mutual information in Ising models. Here we take a more conceptual approach to proving lower bounds on the mutual information. Our proof generalizes well beyond Ising models, to arbitrary Markov random fields with higher order interactions. As an application, we obtain algorithms for learning Markov random fields on bounded degree graphs on n nodes with r-order interactions in n r time and log n sample complexity. Our algorithms also extend to various partial observation models. |
| first_indexed | 2024-09-23T12:33:06Z |
| format | Article |
| id | mit-1721.1/116218 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T12:33:06Z |
| publishDate | 2018 |
| record_format | dspace |
| spelling | mit-1721.1/1162182022-09-28T08:33:29Z Information theoretic properties of Markov Random Fields, and their algorithmic applications Hamilton, Linus Ulysses Koehler, Frederic Moitra, Ankur Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Mathematics Hamilton, Linus Ulysses Koehler, Frederic Moitra, Ankur Markov random fields are a popular model for high-dimensional probability distributions. Over the years, many mathematical, statistical and algorithmic problems on them have been studied. Until recently, the only known algorithms for provably learning them relied on exhaustive search, correlation decay or various incoherence assumptions. Bresler [1] gave an algorithm for learning general Ising models on bounded degree graphs. His approach was based on a structural result about mutual information in Ising models. Here we take a more conceptual approach to proving lower bounds on the mutual information. Our proof generalizes well beyond Ising models, to arbitrary Markov random fields with higher order interactions. As an application, we obtain algorithms for learning Markov random fields on bounded degree graphs on n nodes with r-order interactions in n r time and log n sample complexity. Our algorithms also extend to various partial observation models. 2018-06-11T18:02:46Z 2018-06-11T18:02:46Z 2016-05 2016-03 2018-05-29T13:46:40Z Article http://purl.org/eprint/type/ConferencePaper 1049-5258 http://hdl.handle.net/1721.1/116218 Hamilton, Linus, Fredderic Koehler and Ankur Moitra. "Information Theoretic Properties of Markov Random Fields, and their Algorithmic Applications." Advances in Neural Information Processing Systems 30 (NIPS 2017). https://orcid.org/0000-0001-7047-0495 https://papers.nips.cc/paper/6840-information-theoretic-properties-of-markov-random-fields-and-their-algorithmic-applications Advances in neural information processing systems Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Neural Information Processing Systems (NIPS) |
| spellingShingle | Hamilton, Linus Ulysses Koehler, Frederic Moitra, Ankur Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title | Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title_full | Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title_fullStr | Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title_full_unstemmed | Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title_short | Information theoretic properties of Markov Random Fields, and their algorithmic applications |
| title_sort | information theoretic properties of markov random fields and their algorithmic applications |
| url | http://hdl.handle.net/1721.1/116218 https://orcid.org/0000-0001-7047-0495 |
| work_keys_str_mv | AT hamiltonlinusulysses informationtheoreticpropertiesofmarkovrandomfieldsandtheiralgorithmicapplications AT koehlerfrederic informationtheoreticpropertiesofmarkovrandomfieldsandtheiralgorithmicapplications AT moitraankur informationtheoreticpropertiesofmarkovrandomfieldsandtheiralgorithmicapplications |