Algorithms above the noise floor
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2018
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/118098 |
| _version_ | 1826210695868317696 |
|---|---|
| author | Schmidt. Ludwig, Ph. D. Massachusetts Institute of Technology |
| author2 | Piotr Indyk. |
| author_facet | Piotr Indyk. Schmidt. Ludwig, Ph. D. Massachusetts Institute of Technology |
| author_sort | Schmidt. Ludwig, Ph. D. Massachusetts Institute of Technology |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018. |
| first_indexed | 2024-09-23T14:53:43Z |
| format | Thesis |
| id | mit-1721.1/118098 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T14:53:43Z |
| publishDate | 2018 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1180982019-04-11T12:29:41Z Algorithms above the noise floor Schmidt. Ludwig, Ph. D. Massachusetts Institute of Technology Piotr Indyk. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Electrical Engineering and Computer Science. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018. Cataloged from PDF version of thesis. Includes bibliographical references (pages 281-297). Many success stories in the data sciences share an intriguing computational phenomenon. While the core algorithmic problems might seem intractable at first, simple heuristics or approximation algorithms often perform surprisingly well in practice. Common examples include optimizing non-convex functions or optimizing over non-convex sets. In theory, such problems are usually NP-hard. But in practice, they are often solved sufficiently well for applications in machine learning and statistics. Even when a problem is convex, we often settle for sub-optimal solutions returned by inexact methods like stochastic gradient descent. And in nearest neighbor search, a variety of approximation algorithms works remarkably well despite the "curse of dimensionality". In this thesis, we study this phenomenon in the context of three fundamental algorithmic problems arising in the data sciences. * In constrained optimization, we show that it is possible to optimize over a wide range of non-convex sets up to the statistical noise floor. * In unconstrained optimization, we prove that important convex problems already require approximation if we want to find a solution quickly. * In nearest neighbor search, we show that approximation guarantees can explain much of the good performance observed in practice. The overarching theme is that the computational hardness of many problems emerges only below the inherent "noise floor" of real data. Hence computational hardness of these problems does not prevent us from finding answers that perform well from a statistical perspective. This offers an explanation for why algorithmic problems in the data sciences often turn out to be easier than expected. by Ludwig Schmidt. Ph. D. 2018-09-17T15:57:26Z 2018-09-17T15:57:26Z 2018 2018 Thesis http://hdl.handle.net/1721.1/118098 1052124168 eng MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 297 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Schmidt. Ludwig, Ph. D. Massachusetts Institute of Technology Algorithms above the noise floor |
| title | Algorithms above the noise floor |
| title_full | Algorithms above the noise floor |
| title_fullStr | Algorithms above the noise floor |
| title_full_unstemmed | Algorithms above the noise floor |
| title_short | Algorithms above the noise floor |
| title_sort | algorithms above the noise floor |
| topic | Electrical Engineering and Computer Science. |
| url | http://hdl.handle.net/1721.1/118098 |
| work_keys_str_mv | AT schmidtludwigphdmassachusettsinstituteoftechnology algorithmsabovethenoisefloor |