Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques
Smart devices, collectively, have very valuable and real-time data which can be used to train very efficient deep learning models for AI applications. However, due to the sensitive nature of this data, people are more concerned about the privacy of their data and not willing to share it. Therefore,...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | http://eprints.usm.my/60077/1/ZAHID%20IQBAL%20-%20TESIS%20cut.pdf |
| _version_ | 1797014402366439424 |
|---|---|
| author | Iqbal, Zahid |
| author_facet | Iqbal, Zahid |
| author_sort | Iqbal, Zahid |
| collection | USM |
| description | Smart devices, collectively, have very valuable and real-time data which can be used to train very efficient deep learning models for AI applications. However, due to the sensitive nature of this data, people are more concerned about the privacy of their data and not willing to share it. Therefore, there is a need to learn from this valuable data in a decentralized fashion by withholding data localized on these intended devices and efficiently performing necessary computation on these devices by exploiting their computational resources. Statistical heterogeneity and fully model heterogeneity are among the key challenges in applying the Decentralized Learning (DL) approaches in real scenarios. Typically, all existing DL techniques assume that all devices would have homogeneous model architecture. However, in real applications of DL, due to different computational resources and distinct business needs of devices, it is intuitive that they may have completely different model architectures. Very limited work has been performed to address fully model heterogeneity problem. In the same way, some work has been performed to address the statistical heterogeneity however mostly is hard to apply in real scenarios or is only for limited use cases. |
| first_indexed | 2024-04-09T03:52:55Z |
| format | Thesis |
| id | usm.eprints-60077 |
| institution | Universiti Sains Malaysia |
| language | English |
| last_indexed | 2024-04-09T03:52:55Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | usm.eprints-600772024-03-07T08:03:06Z http://eprints.usm.my/60077/ Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques Iqbal, Zahid QA75.5-76.95 Electronic computers. Computer science Smart devices, collectively, have very valuable and real-time data which can be used to train very efficient deep learning models for AI applications. However, due to the sensitive nature of this data, people are more concerned about the privacy of their data and not willing to share it. Therefore, there is a need to learn from this valuable data in a decentralized fashion by withholding data localized on these intended devices and efficiently performing necessary computation on these devices by exploiting their computational resources. Statistical heterogeneity and fully model heterogeneity are among the key challenges in applying the Decentralized Learning (DL) approaches in real scenarios. Typically, all existing DL techniques assume that all devices would have homogeneous model architecture. However, in real applications of DL, due to different computational resources and distinct business needs of devices, it is intuitive that they may have completely different model architectures. Very limited work has been performed to address fully model heterogeneity problem. In the same way, some work has been performed to address the statistical heterogeneity however mostly is hard to apply in real scenarios or is only for limited use cases. 2023-07 Thesis NonPeerReviewed application/pdf en http://eprints.usm.my/60077/1/ZAHID%20IQBAL%20-%20TESIS%20cut.pdf Iqbal, Zahid (2023) Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques. PhD thesis, Universiti Sains Malaysia. |
| spellingShingle | QA75.5-76.95 Electronic computers. Computer science Iqbal, Zahid Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title | Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title_full | Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title_fullStr | Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title_full_unstemmed | Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title_short | Communication Efficient Decentralized Collaborative Learning Of Heterogeneous Deep Learning Models Using Distillation And Incentive-Based Techniques |
| title_sort | communication efficient decentralized collaborative learning of heterogeneous deep learning models using distillation and incentive based techniques |
| topic | QA75.5-76.95 Electronic computers. Computer science |
| url | http://eprints.usm.my/60077/1/ZAHID%20IQBAL%20-%20TESIS%20cut.pdf |
| work_keys_str_mv | AT iqbalzahid communicationefficientdecentralizedcollaborativelearningofheterogeneousdeeplearningmodelsusingdistillationandincentivebasedtechniques |