Learning with generalized negative dependence : probabilistic models of diversity for machine learning
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/1721.1/122739 |
| _version_ | 1826188061300490240 |
|---|---|
| author | Mariet, Zelda Elaine. |
| author2 | Suvrit Sra. |
| author_facet | Suvrit Sra. Mariet, Zelda Elaine. |
| author_sort | Mariet, Zelda Elaine. |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 |
| first_indexed | 2024-09-23T07:54:00Z |
| format | Thesis |
| id | mit-1721.1/122739 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T07:54:00Z |
| publishDate | 2019 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1227392019-11-22T03:51:58Z Learning with generalized negative dependence : probabilistic models of diversity for machine learning Probabilistic models of diversity for machine learning Mariet, Zelda Elaine. Suvrit Sra. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Electrical Engineering and Computer Science. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 Cataloged from PDF version of thesis. Includes bibliographical references (pages 139-150). This thesis establishes negative dependence as a powerful and computationally efficient framework to analyze machine learning problems that require a theoretical model of diversification. Examples of such problems include experimental design and model compression: subset-selection problems that require carefully balancing the quality of each selected element with the diversity of the subset as a whole. Negative dependence, which models the behavior of "repelling" random variables, provides a rich mathematical framework for the analysis of such problems. Leveraging negative dependence theory for machine learning requires (a) scalable sampling and learning algorithms for negatively dependent measures, and (b) negatively dependent measures able to model the specific diversity requirements that arise in machine learning. These problems are the focus of this thesis. The first part of this thesis develops scalable sampling and learning algorithms for determinantal point processes (DPPs), popular negatively dependent measures with many applications to machine learning. For scalable sampling, we introduce a theoretically-motivated generative deep neural network for DPP-like samples over arbitrary ground sets. To address the learning problem, we show that algorithms for maximum likelihood estimation (MLE) for DPps are drastically sped up with Kronecker kernels, and that MLE can be further enriched by negative samples. The second part of this thesis leverages negative dependence for core problems in machine learning. We begin by deriving a generalized form of volume sampling (GVS) based on elementary symmetric polynomials, and prove that the induced measures exhibit strong negative dependence properties. We then show that classical forms of optimal experimental design can be cast as optimization problems based on GVS, for which we derive randomized and greedy algorithms to obtain the associated designs. Finally, we introduce exponentiated strongly Rayleigh measures, which allow for simple tuning of the strength of repulsive forces between similar items while still enjoying fast sampling algorithms. The great flexibility of exponentiated strongly Rayleigh measures makes them an ideal tool for machine learning problems that benefit from negative dependence theory. by Zelda E. Lawson Mariet. Ph. D. Ph.D. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science 2019-11-04T20:21:55Z 2019-11-04T20:21:55Z 2019 2019 Thesis https://hdl.handle.net/1721.1/122739 1124746104 eng MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 157 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Mariet, Zelda Elaine. Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title | Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title_full | Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title_fullStr | Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title_full_unstemmed | Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title_short | Learning with generalized negative dependence : probabilistic models of diversity for machine learning |
| title_sort | learning with generalized negative dependence probabilistic models of diversity for machine learning |
| topic | Electrical Engineering and Computer Science. |
| url | https://hdl.handle.net/1721.1/122739 |
| work_keys_str_mv | AT marietzeldaelaine learningwithgeneralizednegativedependenceprobabilisticmodelsofdiversityformachinelearning AT marietzeldaelaine probabilisticmodelsofdiversityformachinelearning |