Message Passing for Maximum Weight Independent Set
In this paper, we investigate the use of message-passing algorithms for the problem of finding the max-weight independent set (MWIS) in a graph. First, we study the performance of the classical loopy max-product belief propagation. We show that each fixed-point estimate of max product can be mapped...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
2011
|
| Online Access: | http://hdl.handle.net/1721.1/62216 https://orcid.org/0000-0003-0737-3259 https://orcid.org/0000-0003-0149-5888 |
| _version_ | 1826210906221051904 |
|---|---|
| author | Sanghavi, Sujay Shah, Devavrat Willsky, Alan S. |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Sanghavi, Sujay Shah, Devavrat Willsky, Alan S. |
| author_sort | Sanghavi, Sujay |
| collection | MIT |
| description | In this paper, we investigate the use of message-passing algorithms for the problem of finding the max-weight independent set (MWIS) in a graph. First, we study the performance of the classical loopy max-product belief propagation. We show that each fixed-point estimate of max product can be mapped in a natural way to an extreme point of the linear programming (LP) polytope associated with the MWIS problem. However, this extreme point may not be the one that maximizes the value of node weights; the particular extreme point at final convergence depends on the initialization of max product. We then show that if max product is started from the natural initialization of uninformative messages, it always solves the correct LP, if it converges. This result is obtained via a direct analysis of the iterative algorithm, and cannot be obtained by looking only at fixed points. The tightness of the LP relaxation is thus necessary for max-product optimality, but it is not sufficient. Motivated by this observation, we show that a simple modification of max product becomes gradient descent on (a smoothed version of) the dual of the LP, and converges to the dual optimum. We also develop a message-passing algorithm that recovers the primal MWIS solution from the output of the descent algorithm. We show that the MWIS estimate obtained using these two algorithms in conjunction is correct when the graph is bipartite and the MWIS is unique. Finally, we show that any problem of maximum a posteriori (MAP) estimation for probability distributions over finite domains can be reduced to an MWIS problem. We believe this reduction will yield new insights and algorithms for MAP estimation. |
| first_indexed | 2024-09-23T14:57:20Z |
| format | Article |
| id | mit-1721.1/62216 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:57:20Z |
| publishDate | 2011 |
| record_format | dspace |
| spelling | mit-1721.1/622162022-09-29T11:40:48Z Message Passing for Maximum Weight Independent Set Sanghavi, Sujay Shah, Devavrat Willsky, Alan S. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Willsky, Alan S. Shah, Devavrat Willsky, Alan S. In this paper, we investigate the use of message-passing algorithms for the problem of finding the max-weight independent set (MWIS) in a graph. First, we study the performance of the classical loopy max-product belief propagation. We show that each fixed-point estimate of max product can be mapped in a natural way to an extreme point of the linear programming (LP) polytope associated with the MWIS problem. However, this extreme point may not be the one that maximizes the value of node weights; the particular extreme point at final convergence depends on the initialization of max product. We then show that if max product is started from the natural initialization of uninformative messages, it always solves the correct LP, if it converges. This result is obtained via a direct analysis of the iterative algorithm, and cannot be obtained by looking only at fixed points. The tightness of the LP relaxation is thus necessary for max-product optimality, but it is not sufficient. Motivated by this observation, we show that a simple modification of max product becomes gradient descent on (a smoothed version of) the dual of the LP, and converges to the dual optimum. We also develop a message-passing algorithm that recovers the primal MWIS solution from the output of the descent algorithm. We show that the MWIS estimate obtained using these two algorithms in conjunction is correct when the graph is bipartite and the MWIS is unique. Finally, we show that any problem of maximum a posteriori (MAP) estimation for probability distributions over finite domains can be reduced to an MWIS problem. We believe this reduction will yield new insights and algorithms for MAP estimation. National Science Foundation (U.S.) (Project CNS 0546590) (Project HSD 0729361) (Project TF 0728554) United States. Army Research Office. Multidisciplinary University Research Initiative (Grant W911NF-06-1-0076) United States. Air Force Office of Scientific Research (Grant Grant FA9550-06-1-0324) 2011-04-15T17:59:24Z 2011-04-15T17:59:24Z 2009-10 2009-04 Article http://purl.org/eprint/type/JournalArticle 0018-9448 INSPEC Accession Number: 10929394 http://hdl.handle.net/1721.1/62216 Sanghavi, S., D. Shah, and A.S. Willsky. “Message Passing for Maximum Weight Independent Set.” Information Theory, IEEE Transactions On 55.11 (2009) : 4822-4834. Copyright © 2009, IEEE https://orcid.org/0000-0003-0737-3259 https://orcid.org/0000-0003-0149-5888 en_US http://dx.doi.org/10.1109/tit.2009.2030448 IEEE transactions on information theory Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf IEEE |
| spellingShingle | Sanghavi, Sujay Shah, Devavrat Willsky, Alan S. Message Passing for Maximum Weight Independent Set |
| title | Message Passing for Maximum Weight Independent Set |
| title_full | Message Passing for Maximum Weight Independent Set |
| title_fullStr | Message Passing for Maximum Weight Independent Set |
| title_full_unstemmed | Message Passing for Maximum Weight Independent Set |
| title_short | Message Passing for Maximum Weight Independent Set |
| title_sort | message passing for maximum weight independent set |
| url | http://hdl.handle.net/1721.1/62216 https://orcid.org/0000-0003-0737-3259 https://orcid.org/0000-0003-0149-5888 |
| work_keys_str_mv | AT sanghavisujay messagepassingformaximumweightindependentset AT shahdevavrat messagepassingformaximumweightindependentset AT willskyalans messagepassingformaximumweightindependentset |