Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics
Abstract The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision‐making such as in climate change applications. We address both...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Geophysical Union (AGU)
2022-11-01
|
| Series: | Journal of Advances in Modeling Earth Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1029/2022MS003162 |
| _version_ | 1797253655286513664 |
|---|---|
| author | Mariana C. A. Clare Maike Sonnewald Redouane Lguensat Julie Deshayes V. Balaji |
| author_facet | Mariana C. A. Clare Maike Sonnewald Redouane Lguensat Julie Deshayes V. Balaji |
| author_sort | Mariana C. A. Clare |
| collection | DOAJ |
| description | Abstract The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision‐making such as in climate change applications. We address both issues by successfully implementing a Bayesian Neural Network (BNN), where parameters are distributions rather than deterministic, and applying novel implementations of explainable AI (XAI) techniques. The uncertainty analysis from the BNN provides a comprehensive overview of the prediction more suited to practitioners' needs than predictions from a classical neural network. Using a BNN means we can calculate the entropy (i.e., uncertainty) of the predictions and determine if the probability of an outcome is statistically significant. To enhance trustworthiness, we also spatially apply the two XAI techniques of Layer‐wise Relevance Propagation (LRP) and SHapley Additive exPlanation (SHAP) values. These XAI methods reveal the extent to which the BNN is suitable and/or trustworthy. Using two techniques gives a more holistic view of BNN skill and its uncertainty, as LRP considers neural network parameters, whereas SHAP considers changes to outputs. We verify these techniques using comparison with intuition from physical theory. The differences in explanation identify potential areas where new physical theory guided studies are needed. |
| first_indexed | 2024-04-24T21:37:31Z |
| format | Article |
| id | doaj.art-878d46663d5e47198bac009170510a97 |
| institution | Directory Open Access Journal |
| issn | 1942-2466 |
| language | English |
| last_indexed | 2024-04-24T21:37:31Z |
| publishDate | 2022-11-01 |
| publisher | American Geophysical Union (AGU) |
| record_format | Article |
| series | Journal of Advances in Modeling Earth Systems |
| spelling | doaj.art-878d46663d5e47198bac009170510a972024-03-21T18:32:29ZengAmerican Geophysical Union (AGU)Journal of Advances in Modeling Earth Systems1942-24662022-11-011411n/an/a10.1029/2022MS003162Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean DynamicsMariana C. A. Clare0Maike Sonnewald1Redouane Lguensat2Julie Deshayes3V. Balaji4Imperial College London London UKProgram in Atmospheric and Oceanic Sciences Princeton University Princeton NJ USAInstitut Pierre‐Simon Laplace IRD Sorbonne Université Paris FranceLOCEAN‐IPSL CNRS Sorbonne Université Paris FranceProgram in Atmospheric and Oceanic Sciences Princeton University Princeton NJ USAAbstract The trustworthiness of neural networks is often challenged because they lack the ability to express uncertainty and explain their skill. This can be problematic given the increasing use of neural networks in high stakes decision‐making such as in climate change applications. We address both issues by successfully implementing a Bayesian Neural Network (BNN), where parameters are distributions rather than deterministic, and applying novel implementations of explainable AI (XAI) techniques. The uncertainty analysis from the BNN provides a comprehensive overview of the prediction more suited to practitioners' needs than predictions from a classical neural network. Using a BNN means we can calculate the entropy (i.e., uncertainty) of the predictions and determine if the probability of an outcome is statistically significant. To enhance trustworthiness, we also spatially apply the two XAI techniques of Layer‐wise Relevance Propagation (LRP) and SHapley Additive exPlanation (SHAP) values. These XAI methods reveal the extent to which the BNN is suitable and/or trustworthy. Using two techniques gives a more holistic view of BNN skill and its uncertainty, as LRP considers neural network parameters, whereas SHAP considers changes to outputs. We verify these techniques using comparison with intuition from physical theory. The differences in explanation identify potential areas where new physical theory guided studies are needed.https://doi.org/10.1029/2022MS003162deep learningchanging climatedynamical ocean regimesBayesian Neural Networksexplainable AIinterpretability |
| spellingShingle | Mariana C. A. Clare Maike Sonnewald Redouane Lguensat Julie Deshayes V. Balaji Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics Journal of Advances in Modeling Earth Systems deep learning changing climate dynamical ocean regimes Bayesian Neural Networks explainable AI interpretability |
| title | Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics |
| title_full | Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics |
| title_fullStr | Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics |
| title_full_unstemmed | Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics |
| title_short | Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics |
| title_sort | explainable artificial intelligence for bayesian neural networks toward trustworthy predictions of ocean dynamics |
| topic | deep learning changing climate dynamical ocean regimes Bayesian Neural Networks explainable AI interpretability |
| url | https://doi.org/10.1029/2022MS003162 |
| work_keys_str_mv | AT marianacaclare explainableartificialintelligenceforbayesianneuralnetworkstowardtrustworthypredictionsofoceandynamics AT maikesonnewald explainableartificialintelligenceforbayesianneuralnetworkstowardtrustworthypredictionsofoceandynamics AT redouanelguensat explainableartificialintelligenceforbayesianneuralnetworkstowardtrustworthypredictionsofoceandynamics AT juliedeshayes explainableartificialintelligenceforbayesianneuralnetworkstowardtrustworthypredictionsofoceandynamics AT vbalaji explainableartificialintelligenceforbayesianneuralnetworkstowardtrustworthypredictionsofoceandynamics |