Utility–Privacy Trade-Off in Distributed Machine Learning Systems
In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mec...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/9/1299 |
| _version_ | 1797488656140730368 |
|---|---|
| author | Xia Zeng Chuanchuan Yang Bin Dai |
| author_facet | Xia Zeng Chuanchuan Yang Bin Dai |
| author_sort | Xia Zeng |
| collection | DOAJ |
| description | In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mechanism to protect the clients’ local parameters. In this paper, from an information-theoretic point of view, we study the utility–privacy trade-off in DML with the help of the DP mechanism. Specifically, three cases including independent clients’ local parameters with independent DP noise, dependent clients’ local parameters with independent/dependent DP noise are considered. Mutual information and conditional mutual information are used to characterize utility and privacy, respectively. First, we show the relationship between utility and privacy for the three cases. Then, we show the optimal noise variance that achieves the maximal utility under a certain level of privacy. Finally, the results of this paper are further illustrated by numerical results. |
| first_indexed | 2024-03-10T00:05:25Z |
| format | Article |
| id | doaj.art-891ae7052f0d4de5a69bf460644127dc |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T00:05:25Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-891ae7052f0d4de5a69bf460644127dc2023-11-23T16:09:20ZengMDPI AGEntropy1099-43002022-09-01249129910.3390/e24091299Utility–Privacy Trade-Off in Distributed Machine Learning SystemsXia Zeng0Chuanchuan Yang1Bin Dai2School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, ChinaPeng Cheng Laboratory, Shenzhen 518055, ChinaSchool of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, ChinaIn distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mechanism to protect the clients’ local parameters. In this paper, from an information-theoretic point of view, we study the utility–privacy trade-off in DML with the help of the DP mechanism. Specifically, three cases including independent clients’ local parameters with independent DP noise, dependent clients’ local parameters with independent/dependent DP noise are considered. Mutual information and conditional mutual information are used to characterize utility and privacy, respectively. First, we show the relationship between utility and privacy for the three cases. Then, we show the optimal noise variance that achieves the maximal utility under a certain level of privacy. Finally, the results of this paper are further illustrated by numerical results.https://www.mdpi.com/1099-4300/24/9/1299differential privacydistributed machine learningmutual informationGaussian noisetrade-off |
| spellingShingle | Xia Zeng Chuanchuan Yang Bin Dai Utility–Privacy Trade-Off in Distributed Machine Learning Systems Entropy differential privacy distributed machine learning mutual information Gaussian noise trade-off |
| title | Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
| title_full | Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
| title_fullStr | Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
| title_full_unstemmed | Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
| title_short | Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
| title_sort | utility privacy trade off in distributed machine learning systems |
| topic | differential privacy distributed machine learning mutual information Gaussian noise trade-off |
| url | https://www.mdpi.com/1099-4300/24/9/1299 |
| work_keys_str_mv | AT xiazeng utilityprivacytradeoffindistributedmachinelearningsystems AT chuanchuanyang utilityprivacytradeoffindistributedmachinelearningsystems AT bindai utilityprivacytradeoffindistributedmachinelearningsystems |