Fast Multistage Compilation of Machine Learning Computation Graphs
Machine learning applications are increasingly requiring fast and more computational power. Many applications like language models have become so large that they are run on distributed systems in parallel. However, getting into the details of optimally scheduling or even just running machine learnin...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156774 https://orcid.org/0009-0006-9656-1946 |
| _version_ | 1826189035810324480 |
|---|---|
| author | Dighe, Kaustubh |
| author2 | Amarasinghe, Saman |
| author_facet | Amarasinghe, Saman Dighe, Kaustubh |
| author_sort | Dighe, Kaustubh |
| collection | MIT |
| description | Machine learning applications are increasingly requiring fast and more computational power. Many applications like language models have become so large that they are run on distributed systems in parallel. However, getting into the details of optimally scheduling or even just running machine learning models on distributed systems can be a distraction for researchers ideating models. Hence there has been development of abstractions to facilitate running machine learning models in parallel on distributed systems. We present a compiler for the StreamIt language- a language made for abstract signal processing and multicore programming. We use that abstraction as a way to distribute the computation of machine learning models programmed in PyTorch. |
| first_indexed | 2024-09-23T08:08:50Z |
| format | Thesis |
| id | mit-1721.1/156774 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T08:08:50Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1567742024-09-17T04:07:02Z Fast Multistage Compilation of Machine Learning Computation Graphs Dighe, Kaustubh Amarasinghe, Saman Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Machine learning applications are increasingly requiring fast and more computational power. Many applications like language models have become so large that they are run on distributed systems in parallel. However, getting into the details of optimally scheduling or even just running machine learning models on distributed systems can be a distraction for researchers ideating models. Hence there has been development of abstractions to facilitate running machine learning models in parallel on distributed systems. We present a compiler for the StreamIt language- a language made for abstract signal processing and multicore programming. We use that abstraction as a way to distribute the computation of machine learning models programmed in PyTorch. M.Eng. 2024-09-16T13:48:18Z 2024-09-16T13:48:18Z 2024-05 2024-07-11T14:37:05.421Z Thesis https://hdl.handle.net/1721.1/156774 https://orcid.org/0009-0006-9656-1946 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Dighe, Kaustubh Fast Multistage Compilation of Machine Learning Computation Graphs |
| title | Fast Multistage Compilation of Machine Learning
Computation Graphs |
| title_full | Fast Multistage Compilation of Machine Learning
Computation Graphs |
| title_fullStr | Fast Multistage Compilation of Machine Learning
Computation Graphs |
| title_full_unstemmed | Fast Multistage Compilation of Machine Learning
Computation Graphs |
| title_short | Fast Multistage Compilation of Machine Learning
Computation Graphs |
| title_sort | fast multistage compilation of machine learning computation graphs |
| url | https://hdl.handle.net/1721.1/156774 https://orcid.org/0009-0006-9656-1946 |
| work_keys_str_mv | AT dighekaustubh fastmultistagecompilationofmachinelearningcomputationgraphs |