BiG-fed: bilevel optimization enhanced graph-aided federated learning
In federated learning (FL), due to the non-i.i.d. nature of distributedly owned local datasets, personalization is an important design goal. In this paper, we investigate FL scenarios in which data owners are related by a network topology (e.g., traffic prediction based on sensor networks). Existing...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/179049 |
| _version_ | 1811679362577072128 |
|---|---|
| author | Xing, Pengwei Lu, Songtao Wu, Lingfei Yu, Han |
| author2 | College of Computing and Data Science |
| author_facet | College of Computing and Data Science Xing, Pengwei Lu, Songtao Wu, Lingfei Yu, Han |
| author_sort | Xing, Pengwei |
| collection | NTU |
| description | In federated learning (FL), due to the non-i.i.d. nature of distributedly owned local datasets, personalization is an important design goal. In this paper, we investigate FL scenarios in which data owners are related by a network topology (e.g., traffic prediction based on sensor networks). Existing personalized FL approaches cannot take this information into account. To address this limitation, we propose the Bilevel Optimization enhanced Graph-aided Federated Learning (BiG-Fed) approach. The inner weights enable local tasks to evolve towards personalization, and the outer shared weights on the server side target the non-i.i.d problem enabling individual tasks to evolve towards a global constraint space. To the best of our knowledge, BiG-Fed is the first bilevel optimization technique to enable FL approaches to cope with two nested optimization tasks at the FL server and FL clients simultaneously. Theoretical analysis shows that BiG-Fed is guaranteed to converge in an efficient manner. Extensive experiments on both synthetic and real-world data demonstrate significant superior performance of BiG-Fed over seven state-of-the-art methods. |
| first_indexed | 2024-10-01T03:07:57Z |
| format | Journal Article |
| id | ntu-10356/179049 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:07:57Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1790492024-07-18T03:10:11Z BiG-fed: bilevel optimization enhanced graph-aided federated learning Xing, Pengwei Lu, Songtao Wu, Lingfei Yu, Han College of Computing and Data Science School of Computer Science and Engineering Computer and Information Science Bilevel optimization Federated learning In federated learning (FL), due to the non-i.i.d. nature of distributedly owned local datasets, personalization is an important design goal. In this paper, we investigate FL scenarios in which data owners are related by a network topology (e.g., traffic prediction based on sensor networks). Existing personalized FL approaches cannot take this information into account. To address this limitation, we propose the Bilevel Optimization enhanced Graph-aided Federated Learning (BiG-Fed) approach. The inner weights enable local tasks to evolve towards personalization, and the outer shared weights on the server side target the non-i.i.d problem enabling individual tasks to evolve towards a global constraint space. To the best of our knowledge, BiG-Fed is the first bilevel optimization technique to enable FL approaches to cope with two nested optimization tasks at the FL server and FL clients simultaneously. Theoretical analysis shows that BiG-Fed is guaranteed to converge in an efficient manner. Extensive experiments on both synthetic and real-world data demonstrate significant superior performance of BiG-Fed over seven state-of-the-art methods. Agency for Science, Technology and Research (A*STAR) AI Singapore Nanyang Technological University National Research Foundation (NRF) Submitted/Accepted version This research is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-RP-2020-019); the Joint NTU-WeBank Research Centre on Fintech (Award No: NWJ-2020-008); the Nanyang Assistant Professorship (NAP); the RIE 2020 Advanced Manufacturing and Engineering (AME) Programmatic Fund (No. A20G8b0102), Singapore; and Future Communications Research & Development Programme (FCPNTU-RG-2021-014). 2024-07-18T03:10:11Z 2024-07-18T03:10:11Z 2022 Journal Article Xing, P., Lu, S., Wu, L. & Yu, H. (2022). BiG-fed: bilevel optimization enhanced graph-aided federated learning. IEEE Transactions On Big Data. https://dx.doi.org/10.1109/TBDATA.2022.3191439 2332-7790 https://hdl.handle.net/10356/179049 10.1109/TBDATA.2022.3191439 2-s2.0-85135248297 en AISG2-RP-2020-019 NWJ-2020-008 A20G8b0102 FCPNTU-RG-2021-014 IEEE Transactions on Big Data © 2022 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/TBDATA.2022.3191439. application/pdf |
| spellingShingle | Computer and Information Science Bilevel optimization Federated learning Xing, Pengwei Lu, Songtao Wu, Lingfei Yu, Han BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title | BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title_full | BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title_fullStr | BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title_full_unstemmed | BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title_short | BiG-fed: bilevel optimization enhanced graph-aided federated learning |
| title_sort | big fed bilevel optimization enhanced graph aided federated learning |
| topic | Computer and Information Science Bilevel optimization Federated learning |
| url | https://hdl.handle.net/10356/179049 |
| work_keys_str_mv | AT xingpengwei bigfedbileveloptimizationenhancedgraphaidedfederatedlearning AT lusongtao bigfedbileveloptimizationenhancedgraphaidedfederatedlearning AT wulingfei bigfedbileveloptimizationenhancedgraphaidedfederatedlearning AT yuhan bigfedbileveloptimizationenhancedgraphaidedfederatedlearning |