FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering
As contemporary systems are being operated in dynamic situations alternating into decentralized and distributed environments from conventional centralized frameworks, Federated Learning (FL) has been gaining attention for an effective architecture when aggregating the information and correspondingly...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10110974/ |
| _version_ | 1797833224118861824 |
|---|---|
| author | Hunmin Lee Daehee Seo |
| author_facet | Hunmin Lee Daehee Seo |
| author_sort | Hunmin Lee |
| collection | DOAJ |
| description | As contemporary systems are being operated in dynamic situations alternating into decentralized and distributed environments from conventional centralized frameworks, Federated Learning (FL) has been gaining attention for an effective architecture when aggregating the information and correspondingly trained AI model of geographically distributed datasets. However, the convergence of FL suffers from the skewed and biased non-IID local dataset acquired from the heterogeneous environment, which is common in real-world practice. To address this issue, we propose a novel Label-wise clustering algorithm in FL (FedLC) that guarantees the convergence among local clients that hold unique distribution. We theoretically analyze the non-IID attributes that potentially affect the performance, defining the six non-IID scenarios in hierarchical order. Through conducting experiments on the suggested non-IID scenarios, we empirically show that the vanilla FL aggregation model is incapable of gaining stable convergence, generating a biased model. By pre-evaluating the local dataset prior to training models, FedLC determines the potential contributions with respect to trainability in a global view, and adaptively selects the locals to collaboratively take part in while aggregation. Our experimental results show that the FedLC outperforms the state-of-the-art non-IID FL optimization studies, offering a robust convergence in a highly skewed and biased non-IID dataset. |
| first_indexed | 2024-04-09T14:20:57Z |
| format | Article |
| id | doaj.art-b7ec2d87d28542249a90f1002f688685 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T14:20:57Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b7ec2d87d28542249a90f1002f6886852023-05-04T23:00:06ZengIEEEIEEE Access2169-35362023-01-0111420824209510.1109/ACCESS.2023.327151710110974FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise ClusteringHunmin Lee0https://orcid.org/0000-0001-7595-9791Daehee Seo1https://orcid.org/0000-0002-5069-398XDepartment of Computer Science, University of Minnesota, Minneapolis, MN, USADepartment of Artificial Intelligence and Data Engineering, Sangmyung University, Seoul, South KoreaAs contemporary systems are being operated in dynamic situations alternating into decentralized and distributed environments from conventional centralized frameworks, Federated Learning (FL) has been gaining attention for an effective architecture when aggregating the information and correspondingly trained AI model of geographically distributed datasets. However, the convergence of FL suffers from the skewed and biased non-IID local dataset acquired from the heterogeneous environment, which is common in real-world practice. To address this issue, we propose a novel Label-wise clustering algorithm in FL (FedLC) that guarantees the convergence among local clients that hold unique distribution. We theoretically analyze the non-IID attributes that potentially affect the performance, defining the six non-IID scenarios in hierarchical order. Through conducting experiments on the suggested non-IID scenarios, we empirically show that the vanilla FL aggregation model is incapable of gaining stable convergence, generating a biased model. By pre-evaluating the local dataset prior to training models, FedLC determines the potential contributions with respect to trainability in a global view, and adaptively selects the locals to collaboratively take part in while aggregation. Our experimental results show that the FedLC outperforms the state-of-the-art non-IID FL optimization studies, offering a robust convergence in a highly skewed and biased non-IID dataset.https://ieeexplore.ieee.org/document/10110974/Convergence optimization in federated learningfederated learning in heterogenous and non-IID datasetlabel-wise clustering |
| spellingShingle | Hunmin Lee Daehee Seo FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering IEEE Access Convergence optimization in federated learning federated learning in heterogenous and non-IID dataset label-wise clustering |
| title | FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering |
| title_full | FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering |
| title_fullStr | FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering |
| title_full_unstemmed | FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering |
| title_short | FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering |
| title_sort | fedlc optimizing federated learning in non iid data via label wise clustering |
| topic | Convergence optimization in federated learning federated learning in heterogenous and non-IID dataset label-wise clustering |
| url | https://ieeexplore.ieee.org/document/10110974/ |
| work_keys_str_mv | AT hunminlee fedlcoptimizingfederatedlearninginnoniiddatavialabelwiseclustering AT daeheeseo fedlcoptimizingfederatedlearninginnoniiddatavialabelwiseclustering |