An XAI method for convolutional neural networks in self-driving cars.
eXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when usi...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2022-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0267282 |
| _version_ | 1818496377874808832 |
|---|---|
| author | Hong-Sik Kim Inwhee Joe |
| author_facet | Hong-Sik Kim Inwhee Joe |
| author_sort | Hong-Sik Kim |
| collection | DOAJ |
| description | eXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when using deep learning models. It makes the models unreliable in the case of reliability-critical applications. So, it is required to explain how they derived such output. It is a reliability-critical application for self-driving cars because the mistakes made by the computers inside them can lead to critical accidents. So, it is necessary to adopt XAI models in this field. In this paper, we propose an XAI method based on computing and explaining the difference of the output values of the neurons in the last hidden layer of convolutional neural networks. First, we input the original image and some modified images of it. Then we derive output values for each image and compare these values. Then, we introduce the Sensitivity Analysis technique to explain which parts of the original image are needed to distinguish the category. In detail, we divide the image into several parts and fill these parts with shades. First, we compute the influence value on the vector indicating the last hidden layer of the model for each of these parts. Then we draw shades whose darkness is in proportion to the influence values. The experimental results show that our approach for XAI in self-driving cars finds the parts needed to distinguish the category of these images accurately. |
| first_indexed | 2024-12-10T18:32:38Z |
| format | Article |
| id | doaj.art-83039d3c75134f998bf15c037a86297b |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-10T18:32:38Z |
| publishDate | 2022-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-83039d3c75134f998bf15c037a86297b2022-12-22T01:37:54ZengPublic Library of Science (PLoS)PLoS ONE1932-62032022-01-01178e026728210.1371/journal.pone.0267282An XAI method for convolutional neural networks in self-driving cars.Hong-Sik KimInwhee JoeeXplainable Artificial Intelligence (XAI) is a new trend of machine learning. Machine learning models are used to predict or decide something, and they derive output based on a large volume of data set. Here, the problem is that it is hard to know why such prediction was derived, especially when using deep learning models. It makes the models unreliable in the case of reliability-critical applications. So, it is required to explain how they derived such output. It is a reliability-critical application for self-driving cars because the mistakes made by the computers inside them can lead to critical accidents. So, it is necessary to adopt XAI models in this field. In this paper, we propose an XAI method based on computing and explaining the difference of the output values of the neurons in the last hidden layer of convolutional neural networks. First, we input the original image and some modified images of it. Then we derive output values for each image and compare these values. Then, we introduce the Sensitivity Analysis technique to explain which parts of the original image are needed to distinguish the category. In detail, we divide the image into several parts and fill these parts with shades. First, we compute the influence value on the vector indicating the last hidden layer of the model for each of these parts. Then we draw shades whose darkness is in proportion to the influence values. The experimental results show that our approach for XAI in self-driving cars finds the parts needed to distinguish the category of these images accurately.https://doi.org/10.1371/journal.pone.0267282 |
| spellingShingle | Hong-Sik Kim Inwhee Joe An XAI method for convolutional neural networks in self-driving cars. PLoS ONE |
| title | An XAI method for convolutional neural networks in self-driving cars. |
| title_full | An XAI method for convolutional neural networks in self-driving cars. |
| title_fullStr | An XAI method for convolutional neural networks in self-driving cars. |
| title_full_unstemmed | An XAI method for convolutional neural networks in self-driving cars. |
| title_short | An XAI method for convolutional neural networks in self-driving cars. |
| title_sort | xai method for convolutional neural networks in self driving cars |
| url | https://doi.org/10.1371/journal.pone.0267282 |
| work_keys_str_mv | AT hongsikkim anxaimethodforconvolutionalneuralnetworksinselfdrivingcars AT inwheejoe anxaimethodforconvolutionalneuralnetworksinselfdrivingcars AT hongsikkim xaimethodforconvolutionalneuralnetworksinselfdrivingcars AT inwheejoe xaimethodforconvolutionalneuralnetworksinselfdrivingcars |