Systematic Literature Review on Cost-Efficient Deep Learning
Cloud computing and deep learning, the recent trends in the software industry, have enabled small companies to scale their business up rapidly. However, this growth is not without a cost – deep learning models are related to the heaviest workloads in cloud data centers. When the business...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10122965/ |
| _version_ | 1797733178650132480 |
|---|---|
| author | Antti Klemetti Mikko Raatikainen Lalli Myllyaho Tommi Mikkonen Jukka K. Nurminen |
| author_facet | Antti Klemetti Mikko Raatikainen Lalli Myllyaho Tommi Mikkonen Jukka K. Nurminen |
| author_sort | Antti Klemetti |
| collection | DOAJ |
| description | Cloud computing and deep learning, the recent trends in the software industry, have enabled small companies to scale their business up rapidly. However, this growth is not without a cost – deep learning models are related to the heaviest workloads in cloud data centers. When the business grows, the monetary cost of deep learning in the cloud also grows fast. Deep learning practitioners should be prepared and equipped to limit the growing cost. We emphasize monetary cost instead of computational cost although often the same methods decrease both types of cost. We performed a systematic literature review on the methods to control the cost of deep learning. Our library search resulted in 16,066 papers from three article databases, IEEE Xplore, ACM Digital Library, and Scopus. We narrowed them down to 112 papers that we categorized and summarized. We found that: 1) Optimizing inference has raised more interest than optimizing training. Widely used deep learning libraries already support inference optimization methods, such as quantization, pruning, and teacher-student. 2) The research has been centered around image inputs, and there seems to be a research gap for other types of inputs. 3) The research has been hardware-oriented, and the most typical approach to control the cost of deep learning is based on algorithm-hardware co-design. 4) Offloading some of the processing to client devices is gaining interest and can potentially reduce the monetary cost of deep learning. |
| first_indexed | 2024-03-12T12:24:24Z |
| format | Article |
| id | doaj.art-19ea513561fb46efa9cff93f275de4a1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T12:24:24Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-19ea513561fb46efa9cff93f275de4a12023-08-29T23:00:32ZengIEEEIEEE Access2169-35362023-01-0111901589018010.1109/ACCESS.2023.327543110122965Systematic Literature Review on Cost-Efficient Deep LearningAntti Klemetti0https://orcid.org/0000-0001-6578-6284Mikko Raatikainen1https://orcid.org/0000-0002-2410-0722Lalli Myllyaho2https://orcid.org/0000-0002-0953-9825Tommi Mikkonen3https://orcid.org/0000-0002-8540-9918Jukka K. Nurminen4https://orcid.org/0000-0001-5083-1927Department of Computer Science, Faculty of Science, University of Helsinki, Helsinki, FinlandDepartment of Computer Science, Faculty of Science, University of Helsinki, Helsinki, FinlandDepartment of Computer Science, Faculty of Science, University of Helsinki, Helsinki, FinlandFaculty of Information Technology, University of Jyväskylä, Jyväskylä, FinlandDepartment of Computer Science, Faculty of Science, University of Helsinki, Helsinki, FinlandCloud computing and deep learning, the recent trends in the software industry, have enabled small companies to scale their business up rapidly. However, this growth is not without a cost – deep learning models are related to the heaviest workloads in cloud data centers. When the business grows, the monetary cost of deep learning in the cloud also grows fast. Deep learning practitioners should be prepared and equipped to limit the growing cost. We emphasize monetary cost instead of computational cost although often the same methods decrease both types of cost. We performed a systematic literature review on the methods to control the cost of deep learning. Our library search resulted in 16,066 papers from three article databases, IEEE Xplore, ACM Digital Library, and Scopus. We narrowed them down to 112 papers that we categorized and summarized. We found that: 1) Optimizing inference has raised more interest than optimizing training. Widely used deep learning libraries already support inference optimization methods, such as quantization, pruning, and teacher-student. 2) The research has been centered around image inputs, and there seems to be a research gap for other types of inputs. 3) The research has been hardware-oriented, and the most typical approach to control the cost of deep learning is based on algorithm-hardware co-design. 4) Offloading some of the processing to client devices is gaining interest and can potentially reduce the monetary cost of deep learning.https://ieeexplore.ieee.org/document/10122965/Cloud computingcost-efficiencycost reductiondeep learningdeep neural networkedge offloading |
| spellingShingle | Antti Klemetti Mikko Raatikainen Lalli Myllyaho Tommi Mikkonen Jukka K. Nurminen Systematic Literature Review on Cost-Efficient Deep Learning IEEE Access Cloud computing cost-efficiency cost reduction deep learning deep neural network edge offloading |
| title | Systematic Literature Review on Cost-Efficient Deep Learning |
| title_full | Systematic Literature Review on Cost-Efficient Deep Learning |
| title_fullStr | Systematic Literature Review on Cost-Efficient Deep Learning |
| title_full_unstemmed | Systematic Literature Review on Cost-Efficient Deep Learning |
| title_short | Systematic Literature Review on Cost-Efficient Deep Learning |
| title_sort | systematic literature review on cost efficient deep learning |
| topic | Cloud computing cost-efficiency cost reduction deep learning deep neural network edge offloading |
| url | https://ieeexplore.ieee.org/document/10122965/ |
| work_keys_str_mv | AT anttiklemetti systematicliteraturereviewoncostefficientdeeplearning AT mikkoraatikainen systematicliteraturereviewoncostefficientdeeplearning AT lallimyllyaho systematicliteraturereviewoncostefficientdeeplearning AT tommimikkonen systematicliteraturereviewoncostefficientdeeplearning AT jukkaknurminen systematicliteraturereviewoncostefficientdeeplearning |