Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry
Deep learning has experienced remarkable success in recent years, leading to significant advancements in various fields such as vision, natural language generation, complex game play, as well as solving difficult scientific problems such as predicting protein folding. Despite these successes, tradit...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156614 |
| _version_ | 1811093003682447360 |
|---|---|
| author | Loh, Charlotte Chang Le |
| author2 | Soljačić, Marin |
| author_facet | Soljačić, Marin Loh, Charlotte Chang Le |
| author_sort | Loh, Charlotte Chang Le |
| collection | MIT |
| description | Deep learning has experienced remarkable success in recent years, leading to significant advancements in various fields such as vision, natural language generation, complex game play, as well as solving difficult scientific problems such as predicting protein folding. Despite these successes, traditional deep learning faces fundamental challenges limiting their scalability and effectiveness. These challenges include the necessity for extensive labeled datasets, the lack of trustworthiness due to model overconfidence, and difficulties in generalizing to new, unseen data. In this thesis, our primary goal is to tackle these issues by introducing novel tools and methods that augment traditional deep learning. We explore various strategies for solving the main bottlenecks of traditional deep learning, which includes incorporating prior known symmetries and inductive biases of the problem, utilizing Bayesian and ensemble methods, and leveraging abundance of unlabeled data in a representation learning framework. We discuss and demonstrate practical applications of these novel tools in diverse domains including vision, photonics, material science and neuroscience. |
| first_indexed | 2024-09-23T15:38:01Z |
| format | Thesis |
| id | mit-1721.1/156614 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T15:38:01Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1566142024-09-04T03:08:35Z Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry Loh, Charlotte Chang Le Soljačić, Marin Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Deep learning has experienced remarkable success in recent years, leading to significant advancements in various fields such as vision, natural language generation, complex game play, as well as solving difficult scientific problems such as predicting protein folding. Despite these successes, traditional deep learning faces fundamental challenges limiting their scalability and effectiveness. These challenges include the necessity for extensive labeled datasets, the lack of trustworthiness due to model overconfidence, and difficulties in generalizing to new, unseen data. In this thesis, our primary goal is to tackle these issues by introducing novel tools and methods that augment traditional deep learning. We explore various strategies for solving the main bottlenecks of traditional deep learning, which includes incorporating prior known symmetries and inductive biases of the problem, utilizing Bayesian and ensemble methods, and leveraging abundance of unlabeled data in a representation learning framework. We discuss and demonstrate practical applications of these novel tools in diverse domains including vision, photonics, material science and neuroscience. Ph.D. 2024-09-03T21:11:52Z 2024-09-03T21:11:52Z 2024-05 2024-07-10T13:01:47.715Z Thesis https://hdl.handle.net/1721.1/156614 0000-0003-4587-808X Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Loh, Charlotte Chang Le Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title | Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title_full | Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title_fullStr | Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title_full_unstemmed | Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title_short | Scalable Representation Learning: On Data-scarcity, Uncertainty and Symmetry |
| title_sort | scalable representation learning on data scarcity uncertainty and symmetry |
| url | https://hdl.handle.net/1721.1/156614 |
| work_keys_str_mv | AT lohcharlottechangle scalablerepresentationlearningondatascarcityuncertaintyandsymmetry |