Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service
This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern technology products, but it is also a significant contributor to carbon footprint. We introduce, Clover, a carbon-frien...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
ACM|The International Conference for High Performance Computing, Networking, Storage and Analysis
2023
|
| Online Access: | https://hdl.handle.net/1721.1/153142 |
| _version_ | 1811089894670336000 |
|---|---|
| author | Li, Baolin Samsi, Siddharth Gadepally, Vijay Tiwari, Devesh |
| author2 | Lincoln Laboratory |
| author_facet | Lincoln Laboratory Li, Baolin Samsi, Siddharth Gadepally, Vijay Tiwari, Devesh |
| author_sort | Li, Baolin |
| collection | MIT |
| description | This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern technology products, but it is also a significant contributor to carbon footprint. We introduce, Clover, a carbon-friendly ML inference service runtime system that balances performance, accuracy, and carbon emissions through mixed-quality models and GPU resource partitioning. Our experimental results demonstrate that Clover is effective in substantially reducing carbon emissions while maintaining high accuracy and meeting service level agreement (SLA) targets. |
| first_indexed | 2024-09-23T14:26:48Z |
| format | Article |
| id | mit-1721.1/153142 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:26:48Z |
| publishDate | 2023 |
| publisher | ACM|The International Conference for High Performance Computing, Networking, Storage and Analysis |
| record_format | dspace |
| spelling | mit-1721.1/1531422024-01-23T18:17:42Z Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service Li, Baolin Samsi, Siddharth Gadepally, Vijay Tiwari, Devesh Lincoln Laboratory This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern technology products, but it is also a significant contributor to carbon footprint. We introduce, Clover, a carbon-friendly ML inference service runtime system that balances performance, accuracy, and carbon emissions through mixed-quality models and GPU resource partitioning. Our experimental results demonstrate that Clover is effective in substantially reducing carbon emissions while maintaining high accuracy and meeting service level agreement (SLA) targets. 2023-12-12T14:34:04Z 2023-12-12T14:34:04Z 2023-11-12 2023-12-01T08:46:33Z Article http://purl.org/eprint/type/ConferencePaper 979-8-4007-0109-2 https://hdl.handle.net/1721.1/153142 Li, Baolin, Samsi, Siddharth, Gadepally, Vijay and Tiwari, Devesh. 2023. "Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service." PUBLISHER_POLICY en https://doi.org/10.1145/3581784.3607034 Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. The author(s) application/pdf ACM|The International Conference for High Performance Computing, Networking, Storage and Analysis Association for Computing Machinery |
| spellingShingle | Li, Baolin Samsi, Siddharth Gadepally, Vijay Tiwari, Devesh Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title | Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title_full | Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title_fullStr | Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title_full_unstemmed | Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title_short | Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service |
| title_sort | clover toward sustainable ai with carbon aware machine learning inference service |
| url | https://hdl.handle.net/1721.1/153142 |
| work_keys_str_mv | AT libaolin clovertowardsustainableaiwithcarbonawaremachinelearninginferenceservice AT samsisiddharth clovertowardsustainableaiwithcarbonawaremachinelearninginferenceservice AT gadepallyvijay clovertowardsustainableaiwithcarbonawaremachinelearninginferenceservice AT tiwaridevesh clovertowardsustainableaiwithcarbonawaremachinelearninginferenceservice |