Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data
Privacy regulations and the physical distribution of heterogeneous data are often primary concerns for the development of deep learning models in a medical context. This paper evaluates the feasibility of differentially private federated learning for chest X-ray classification as a defense against d...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/14/5195 |
| _version_ | 1797433147551383552 |
|---|---|
| author | Joceline Ziegler Bjarne Pfitzner Heinrich Schulz Axel Saalbach Bert Arnrich |
| author_facet | Joceline Ziegler Bjarne Pfitzner Heinrich Schulz Axel Saalbach Bert Arnrich |
| author_sort | Joceline Ziegler |
| collection | DOAJ |
| description | Privacy regulations and the physical distribution of heterogeneous data are often primary concerns for the development of deep learning models in a medical context. This paper evaluates the feasibility of differentially private federated learning for chest X-ray classification as a defense against data privacy attacks. To the best of our knowledge, we are the first to directly compare the impact of differentially private training on two different neural network architectures, DenseNet121 and ResNet50. Extending the federated learning environments previously analyzed in terms of privacy, we simulated a heterogeneous and imbalanced federated setting by distributing images from the public CheXpert and Mendeley chest X-ray datasets unevenly among 36 clients. Both non-private baseline models achieved an area under the receiver operating characteristic curve (AUC) of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>94</mn></mrow></semantics></math></inline-formula> on the binary classification task of detecting the presence of a medical finding. We demonstrate that both model architectures are vulnerable to privacy violation by applying image reconstruction attacks to local model updates from individual clients. The attack was particularly successful during later training stages. To mitigate the risk of a privacy breach, we integrated Rényi differential privacy with a Gaussian noise mechanism into local model training. We evaluate model performance and attack vulnerability for privacy budgets <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>ε</mi><mo>∈</mo><mo>{</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>6</mn><mo>,</mo><mn>10</mn><mo>}</mo></mrow></semantics></math></inline-formula>. The DenseNet121 achieved the best utility-privacy trade-off with an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>94</mn></mrow></semantics></math></inline-formula> for <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>ε</mi><mo>=</mo><mn>6</mn></mrow></semantics></math></inline-formula>. Model performance deteriorated slightly for individual clients compared to the non-private baseline. The ResNet50 only reached an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>76</mn></mrow></semantics></math></inline-formula> in the same privacy setting. Its performance was inferior to that of the DenseNet121 for all considered privacy constraints, suggesting that the DenseNet121 architecture is more robust to differentially private training. |
| first_indexed | 2024-03-09T10:12:58Z |
| format | Article |
| id | doaj.art-ea5d8e0281df47e0924d46a86656678b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T10:12:58Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-ea5d8e0281df47e0924d46a86656678b2023-12-01T22:39:58ZengMDPI AGSensors1424-82202022-07-012214519510.3390/s22145195Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray DataJoceline Ziegler0Bjarne Pfitzner1Heinrich Schulz2Axel Saalbach3Bert Arnrich4Digital Engineering Faculty, University of Potsdam, 14482 Potsdam, GermanyDigital Engineering Faculty, University of Potsdam, 14482 Potsdam, GermanyPhilips Research, 22335 Hamburg, GermanyPhilips Research, 22335 Hamburg, GermanyDigital Engineering Faculty, University of Potsdam, 14482 Potsdam, GermanyPrivacy regulations and the physical distribution of heterogeneous data are often primary concerns for the development of deep learning models in a medical context. This paper evaluates the feasibility of differentially private federated learning for chest X-ray classification as a defense against data privacy attacks. To the best of our knowledge, we are the first to directly compare the impact of differentially private training on two different neural network architectures, DenseNet121 and ResNet50. Extending the federated learning environments previously analyzed in terms of privacy, we simulated a heterogeneous and imbalanced federated setting by distributing images from the public CheXpert and Mendeley chest X-ray datasets unevenly among 36 clients. Both non-private baseline models achieved an area under the receiver operating characteristic curve (AUC) of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>94</mn></mrow></semantics></math></inline-formula> on the binary classification task of detecting the presence of a medical finding. We demonstrate that both model architectures are vulnerable to privacy violation by applying image reconstruction attacks to local model updates from individual clients. The attack was particularly successful during later training stages. To mitigate the risk of a privacy breach, we integrated Rényi differential privacy with a Gaussian noise mechanism into local model training. We evaluate model performance and attack vulnerability for privacy budgets <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>ε</mi><mo>∈</mo><mo>{</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>6</mn><mo>,</mo><mn>10</mn><mo>}</mo></mrow></semantics></math></inline-formula>. The DenseNet121 achieved the best utility-privacy trade-off with an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>94</mn></mrow></semantics></math></inline-formula> for <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>ε</mi><mo>=</mo><mn>6</mn></mrow></semantics></math></inline-formula>. Model performance deteriorated slightly for individual clients compared to the non-private baseline. The ResNet50 only reached an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0</mn><mo>.</mo><mn>76</mn></mrow></semantics></math></inline-formula> in the same privacy setting. Its performance was inferior to that of the DenseNet121 for all considered privacy constraints, suggesting that the DenseNet121 architecture is more robust to differentially private training.https://www.mdpi.com/1424-8220/22/14/5195federated learningprivacy and securityprivacy attackX-ray |
| spellingShingle | Joceline Ziegler Bjarne Pfitzner Heinrich Schulz Axel Saalbach Bert Arnrich Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data Sensors federated learning privacy and security privacy attack X-ray |
| title | Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data |
| title_full | Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data |
| title_fullStr | Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data |
| title_full_unstemmed | Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data |
| title_short | Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-ray Data |
| title_sort | defending against reconstruction attacks through differentially private federated learning for classification of heterogeneous chest x ray data |
| topic | federated learning privacy and security privacy attack X-ray |
| url | https://www.mdpi.com/1424-8220/22/14/5195 |
| work_keys_str_mv | AT jocelineziegler defendingagainstreconstructionattacksthroughdifferentiallyprivatefederatedlearningforclassificationofheterogeneouschestxraydata AT bjarnepfitzner defendingagainstreconstructionattacksthroughdifferentiallyprivatefederatedlearningforclassificationofheterogeneouschestxraydata AT heinrichschulz defendingagainstreconstructionattacksthroughdifferentiallyprivatefederatedlearningforclassificationofheterogeneouschestxraydata AT axelsaalbach defendingagainstreconstructionattacksthroughdifferentiallyprivatefederatedlearningforclassificationofheterogeneouschestxraydata AT bertarnrich defendingagainstreconstructionattacksthroughdifferentiallyprivatefederatedlearningforclassificationofheterogeneouschestxraydata |