FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing
Federated learning (FL) is an emerging decentralized machine learning framework enabling private global model training by collaboratively leveraging local client data without transferring it centrally. Unlike traditional distributed optimization, FL trains the model at the local client and then aggr...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/23/12962 |
| _version_ | 1797400360898265088 |
|---|---|
| author | Yankai Lv Haiyan Ding Hao Wu Yiji Zhao Lei Zhang |
| author_facet | Yankai Lv Haiyan Ding Hao Wu Yiji Zhao Lei Zhang |
| author_sort | Yankai Lv |
| collection | DOAJ |
| description | Federated learning (FL) is an emerging decentralized machine learning framework enabling private global model training by collaboratively leveraging local client data without transferring it centrally. Unlike traditional distributed optimization, FL trains the model at the local client and then aggregates it at the server. While this approach reduces communication costs, the local datasets of different clients are non-Independent and Identically Distributed (non-IID), which may make the local model inconsistent. The present study suggests a FL algorithm that leverages regularization and data sharing (FedRDS). The local loss function is adapted by introducing a regularization term in each round of training so that the local model will gradually move closer to the global model. However, when the client data distribution gap becomes large, adding regularization items will increase the degree of client drift. Based on this, we used a data-sharing method in which a portion of server data is taken out as a shared dataset during the initialization. We then evenly distributed these data to each client to mitigate the problem of client drift by reducing the difference in client data distribution. Analysis of experimental outcomes indicates that FedRDS surpasses some known FL methods in various image classification tasks, enhancing both communication efficacy and accuracy. |
| first_indexed | 2024-03-09T01:54:28Z |
| format | Article |
| id | doaj.art-102a48c91f834bb385cadb554d1e8e1b |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T01:54:28Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-102a48c91f834bb385cadb554d1e8e1b2023-12-08T15:12:17ZengMDPI AGApplied Sciences2076-34172023-12-0113231296210.3390/app132312962FedRDS: Federated Learning on Non-IID Data via Regularization and Data SharingYankai Lv0Haiyan Ding1Hao Wu2Yiji Zhao3Lei Zhang4School of Information Science and Engineering, Yunnan University, Kunming 650091, ChinaSchool of Information Science and Engineering, Yunnan University, Kunming 650091, ChinaSchool of Information Science and Engineering, Yunnan University, Kunming 650091, ChinaSchool of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, ChinaSchool of Electrical and Automation Engineering, Nanjing Normal University, Nanjing 210024, ChinaFederated learning (FL) is an emerging decentralized machine learning framework enabling private global model training by collaboratively leveraging local client data without transferring it centrally. Unlike traditional distributed optimization, FL trains the model at the local client and then aggregates it at the server. While this approach reduces communication costs, the local datasets of different clients are non-Independent and Identically Distributed (non-IID), which may make the local model inconsistent. The present study suggests a FL algorithm that leverages regularization and data sharing (FedRDS). The local loss function is adapted by introducing a regularization term in each round of training so that the local model will gradually move closer to the global model. However, when the client data distribution gap becomes large, adding regularization items will increase the degree of client drift. Based on this, we used a data-sharing method in which a portion of server data is taken out as a shared dataset during the initialization. We then evenly distributed these data to each client to mitigate the problem of client drift by reducing the difference in client data distribution. Analysis of experimental outcomes indicates that FedRDS surpasses some known FL methods in various image classification tasks, enhancing both communication efficacy and accuracy.https://www.mdpi.com/2076-3417/13/23/12962federated learningnon-IID dataregularizationdata sharingmachine learning |
| spellingShingle | Yankai Lv Haiyan Ding Hao Wu Yiji Zhao Lei Zhang FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing Applied Sciences federated learning non-IID data regularization data sharing machine learning |
| title | FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing |
| title_full | FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing |
| title_fullStr | FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing |
| title_full_unstemmed | FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing |
| title_short | FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing |
| title_sort | fedrds federated learning on non iid data via regularization and data sharing |
| topic | federated learning non-IID data regularization data sharing machine learning |
| url | https://www.mdpi.com/2076-3417/13/23/12962 |
| work_keys_str_mv | AT yankailv fedrdsfederatedlearningonnoniiddataviaregularizationanddatasharing AT haiyanding fedrdsfederatedlearningonnoniiddataviaregularizationanddatasharing AT haowu fedrdsfederatedlearningonnoniiddataviaregularizationanddatasharing AT yijizhao fedrdsfederatedlearningonnoniiddataviaregularizationanddatasharing AT leizhang fedrdsfederatedlearningonnoniiddataviaregularizationanddatasharing |