Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI
Artificial Intelligence (AI) has achieved remarkable success in image generation, image analysis, and language modeling, making data-driven techniques increasingly relevant in practical real-world applications, promising enhanced creativity and efficiency for human users. However, the deployment of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-07-01
|
| Series: | Frontiers in Bioinformatics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fbinf.2023.1194993/full |
| _version_ | 1797786635608260608 |
|---|---|
| author | Adriano Lucieri Adriano Lucieri Andreas Dengel Andreas Dengel Sheraz Ahmed |
| author_facet | Adriano Lucieri Adriano Lucieri Andreas Dengel Andreas Dengel Sheraz Ahmed |
| author_sort | Adriano Lucieri |
| collection | DOAJ |
| description | Artificial Intelligence (AI) has achieved remarkable success in image generation, image analysis, and language modeling, making data-driven techniques increasingly relevant in practical real-world applications, promising enhanced creativity and efficiency for human users. However, the deployment of AI in high-stakes domains such as infrastructure and healthcare still raises concerns regarding algorithm accountability and safety. The emerging field of explainable AI (XAI) has made significant strides in developing interfaces that enable humans to comprehend the decisions made by data-driven models. Among these approaches, concept-based explainability stands out due to its ability to align explanations with high-level concepts familiar to users. Nonetheless, early research in adversarial machine learning has unveiled that exposing model explanations can render victim models more susceptible to attacks. This is the first study to investigate and compare the impact of concept-based explanations on the privacy of Deep Learning based AI models in the context of biomedical image analysis. An extensive privacy benchmark is conducted on three different state-of-the-art model architectures (ResNet50, NFNet, ConvNeXt) trained on two biomedical (ISIC and EyePACS) and one synthetic dataset (SCDB). The success of membership inference attacks while exposing varying degrees of attribution-based and concept-based explanations is systematically compared. The findings indicate that, in theory, concept-based explanations can potentially increase the vulnerability of a private AI system by up to 16% compared to attributions in the baseline setting. However, it is demonstrated that, in more realistic attack scenarios, the threat posed by explanations is negligible in practice. Furthermore, actionable recommendations are provided to ensure the safe deployment of concept-based XAI systems. In addition, the impact of differential privacy (DP) on the quality of concept-based explanations is explored, revealing that while negatively influencing the explanation ability, DP can have an adverse effect on the models’ privacy. |
| first_indexed | 2024-03-13T01:10:38Z |
| format | Article |
| id | doaj.art-60f8a579ed5a4138990020bce17b91a8 |
| institution | Directory Open Access Journal |
| issn | 2673-7647 |
| language | English |
| last_indexed | 2024-03-13T01:10:38Z |
| publishDate | 2023-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Bioinformatics |
| spelling | doaj.art-60f8a579ed5a4138990020bce17b91a82023-07-05T22:47:40ZengFrontiers Media S.A.Frontiers in Bioinformatics2673-76472023-07-01310.3389/fbinf.2023.11949931194993Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AIAdriano Lucieri0Adriano Lucieri1Andreas Dengel2Andreas Dengel3Sheraz Ahmed4Smart Data and Knowledge Services (SDS), Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH, Kaiserslautern, GermanyComputer Science Department, RPTU Kaiserslautern-Landau, Kaiserslautern, GermanySmart Data and Knowledge Services (SDS), Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH, Kaiserslautern, GermanyComputer Science Department, RPTU Kaiserslautern-Landau, Kaiserslautern, GermanySmart Data and Knowledge Services (SDS), Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH, Kaiserslautern, GermanyArtificial Intelligence (AI) has achieved remarkable success in image generation, image analysis, and language modeling, making data-driven techniques increasingly relevant in practical real-world applications, promising enhanced creativity and efficiency for human users. However, the deployment of AI in high-stakes domains such as infrastructure and healthcare still raises concerns regarding algorithm accountability and safety. The emerging field of explainable AI (XAI) has made significant strides in developing interfaces that enable humans to comprehend the decisions made by data-driven models. Among these approaches, concept-based explainability stands out due to its ability to align explanations with high-level concepts familiar to users. Nonetheless, early research in adversarial machine learning has unveiled that exposing model explanations can render victim models more susceptible to attacks. This is the first study to investigate and compare the impact of concept-based explanations on the privacy of Deep Learning based AI models in the context of biomedical image analysis. An extensive privacy benchmark is conducted on three different state-of-the-art model architectures (ResNet50, NFNet, ConvNeXt) trained on two biomedical (ISIC and EyePACS) and one synthetic dataset (SCDB). The success of membership inference attacks while exposing varying degrees of attribution-based and concept-based explanations is systematically compared. The findings indicate that, in theory, concept-based explanations can potentially increase the vulnerability of a private AI system by up to 16% compared to attributions in the baseline setting. However, it is demonstrated that, in more realistic attack scenarios, the threat posed by explanations is negligible in practice. Furthermore, actionable recommendations are provided to ensure the safe deployment of concept-based XAI systems. In addition, the impact of differential privacy (DP) on the quality of concept-based explanations is explored, revealing that while negatively influencing the explanation ability, DP can have an adverse effect on the models’ privacy.https://www.frontiersin.org/articles/10.3389/fbinf.2023.1194993/fulldeep learningbiomedical image analysisexplainable AIconcept-based explainabilityattribution mapsadversarial machine learning |
| spellingShingle | Adriano Lucieri Adriano Lucieri Andreas Dengel Andreas Dengel Sheraz Ahmed Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI Frontiers in Bioinformatics deep learning biomedical image analysis explainable AI concept-based explainability attribution maps adversarial machine learning |
| title | Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI |
| title_full | Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI |
| title_fullStr | Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI |
| title_full_unstemmed | Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI |
| title_short | Translating theory into practice: assessing the privacy implications of concept-based explanations for biomedical AI |
| title_sort | translating theory into practice assessing the privacy implications of concept based explanations for biomedical ai |
| topic | deep learning biomedical image analysis explainable AI concept-based explainability attribution maps adversarial machine learning |
| url | https://www.frontiersin.org/articles/10.3389/fbinf.2023.1194993/full |
| work_keys_str_mv | AT adrianolucieri translatingtheoryintopracticeassessingtheprivacyimplicationsofconceptbasedexplanationsforbiomedicalai AT adrianolucieri translatingtheoryintopracticeassessingtheprivacyimplicationsofconceptbasedexplanationsforbiomedicalai AT andreasdengel translatingtheoryintopracticeassessingtheprivacyimplicationsofconceptbasedexplanationsforbiomedicalai AT andreasdengel translatingtheoryintopracticeassessingtheprivacyimplicationsofconceptbasedexplanationsforbiomedicalai AT sherazahmed translatingtheoryintopracticeassessingtheprivacyimplicationsofconceptbasedexplanationsforbiomedicalai |