Computational Hardness in Random Optimization Problems from the Overlap Gap Property
We study the limits of efficient algorithms in random optimization problems. In these problems, we are given a random objective function and our goal is to find an input achieving a large output. These problems often exhibit information-computation gaps, where the maximum objective that exists is la...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/143164 |
| _version_ | 1826215145342238720 |
|---|---|
| author | Huang, Brice |
| author2 | Bresler, Guy |
| author_facet | Bresler, Guy Huang, Brice |
| author_sort | Huang, Brice |
| collection | MIT |
| description | We study the limits of efficient algorithms in random optimization problems. In these problems, we are given a random objective function and our goal is to find an input achieving a large output. These problems often exhibit information-computation gaps, where the maximum objective that exists is larger than the maximum objective that known efficient algorithms can find. Our goal is to find rigorous evidence of computational hardness in the hard regime.
We focus on the problems of random k-SAT and mean-field spin glasses. Our results are:
• It is known that random k-SAT has a satisfying assignment with high probability up to clause density [formula], while the best known algorithm (Fix) finds a satisfying assignment up to clause density [formula]. We prove that low degree polynomial algorithms cannot find a satisfying assignment above clause density [formula], for a universal constant κ∗ ≈ 4.911. Low degree polynomial algorithms encompass Fix, message passing algorithms including Belief and Survey Propagation guided decimation, and local algorithms on the factor graph. This is the first hardness result against any class of algorithms within a constant factor of the clause density achieved by Fix.
• The maximum asymptotic value OPT of the Hamiltonian [formula] of a spherical or Ising mixed p-spin glass is given by the celebrated Parisi formula. Recently developed approximate message passing algorithms efficiently optimize [formula] up to a value ALG given by an extended Parisi formula, which minimizes over a larger space of non-monotone functional order parameters. These two objectives coincide for spin glasses exhibiting a no overlap gap property, but are generically not equal. We prove that for mixed even p-spin models, no algorithm satisfying an overlap concentration property can produce an objective larger than ALG. This property holds for all algorithms with suitably Lipschitz dependence on the disorder coefficients of HN , including natural formulations of gradient descent, approximate message passing, and Langevin dynamics run for bounded time. In particular, this includes the algorithms achieving ALG.
We prove these results by extending the overlap gap property (OGP) framework of Gamarnik and Sudan to multi-OGPs, which consider forbidden constellations containing several solutions. Our results for random k-SAT are proved by a multi-OGP that generalizes the ladder constellation introduced by Wein. Our results for spin glasses are proved by a new multi-OGP, the branching OGP, that uses an arbitrarily complex ultrametric constellation of solutions. |
| first_indexed | 2024-09-23T16:17:17Z |
| format | Thesis |
| id | mit-1721.1/143164 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:17:17Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1431642022-06-16T03:03:38Z Computational Hardness in Random Optimization Problems from the Overlap Gap Property Huang, Brice Bresler, Guy Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science We study the limits of efficient algorithms in random optimization problems. In these problems, we are given a random objective function and our goal is to find an input achieving a large output. These problems often exhibit information-computation gaps, where the maximum objective that exists is larger than the maximum objective that known efficient algorithms can find. Our goal is to find rigorous evidence of computational hardness in the hard regime. We focus on the problems of random k-SAT and mean-field spin glasses. Our results are: • It is known that random k-SAT has a satisfying assignment with high probability up to clause density [formula], while the best known algorithm (Fix) finds a satisfying assignment up to clause density [formula]. We prove that low degree polynomial algorithms cannot find a satisfying assignment above clause density [formula], for a universal constant κ∗ ≈ 4.911. Low degree polynomial algorithms encompass Fix, message passing algorithms including Belief and Survey Propagation guided decimation, and local algorithms on the factor graph. This is the first hardness result against any class of algorithms within a constant factor of the clause density achieved by Fix. • The maximum asymptotic value OPT of the Hamiltonian [formula] of a spherical or Ising mixed p-spin glass is given by the celebrated Parisi formula. Recently developed approximate message passing algorithms efficiently optimize [formula] up to a value ALG given by an extended Parisi formula, which minimizes over a larger space of non-monotone functional order parameters. These two objectives coincide for spin glasses exhibiting a no overlap gap property, but are generically not equal. We prove that for mixed even p-spin models, no algorithm satisfying an overlap concentration property can produce an objective larger than ALG. This property holds for all algorithms with suitably Lipschitz dependence on the disorder coefficients of HN , including natural formulations of gradient descent, approximate message passing, and Langevin dynamics run for bounded time. In particular, this includes the algorithms achieving ALG. We prove these results by extending the overlap gap property (OGP) framework of Gamarnik and Sudan to multi-OGPs, which consider forbidden constellations containing several solutions. Our results for random k-SAT are proved by a multi-OGP that generalizes the ladder constellation introduced by Wein. Our results for spin glasses are proved by a new multi-OGP, the branching OGP, that uses an arbitrarily complex ultrametric constellation of solutions. S.M. 2022-06-15T13:00:42Z 2022-06-15T13:00:42Z 2022-02 2022-03-04T20:59:42.066Z Thesis https://hdl.handle.net/1721.1/143164 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Huang, Brice Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title | Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title_full | Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title_fullStr | Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title_full_unstemmed | Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title_short | Computational Hardness in Random Optimization Problems from the Overlap Gap Property |
| title_sort | computational hardness in random optimization problems from the overlap gap property |
| url | https://hdl.handle.net/1721.1/143164 |
| work_keys_str_mv | AT huangbrice computationalhardnessinrandomoptimizationproblemsfromtheoverlapgapproperty |