Ontology Concept Extraction Algorithm for Deep Neural Networks
An important drawback of deep neural networks limiting their application in critical tasks is the lack of explainability. Recently, several methods have been proposed to explain and interpret the results obtained by deep neural networks, however, the majority of these methods are targeted mostly at...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
FRUCT
2022-11-01
|
| Series: | Proceedings of the XXth Conference of Open Innovations Association FRUCT |
| Subjects: | |
| Online Access: | https://www.fruct.org/publications/volume-32/fruct32/files/Pon.pdf |
| _version_ | 1797986908152791040 |
|---|---|
| author | Andrew Ponomarev Anton Agafonov |
| author_facet | Andrew Ponomarev Anton Agafonov |
| author_sort | Andrew Ponomarev |
| collection | DOAJ |
| description | An important drawback of deep neural networks limiting their application in critical tasks is the lack of explainability. Recently, several methods have been proposed to explain and interpret the results obtained by deep neural networks, however, the majority of these methods are targeted mostly at AI experts. Ontology-based explanation techniques seem promising, as they can be used to form explanations using domain terms (corresponding to ontology concepts) and logical statements, which is more understandable by domain experts. Recently, it has been shown, that inner representations (layer activations) of deep neural network can often be aligned with ontology concepts. However, not every concept can be matched with the output of every layer, and it can be computationally hard to identify the particular layer that can be easily aligned with the given concept, which is aggravated by the number of concepts in a typical ontology. The paper proposes an algorithm to address this problem. For each ontology concept it helps to identify neural network layer, which produces output that can be best aligned with the given concept. These connections can then be used to identify all the ontology concepts relevant to the sample and explain the network output in a user-friendly way. |
| first_indexed | 2024-04-11T07:40:32Z |
| format | Article |
| id | doaj.art-315c8ac8e068489090c6c00a24574add |
| institution | Directory Open Access Journal |
| issn | 2305-7254 2343-0737 |
| language | English |
| last_indexed | 2024-04-11T07:40:32Z |
| publishDate | 2022-11-01 |
| publisher | FRUCT |
| record_format | Article |
| series | Proceedings of the XXth Conference of Open Innovations Association FRUCT |
| spelling | doaj.art-315c8ac8e068489090c6c00a24574add2022-12-22T04:36:35ZengFRUCTProceedings of the XXth Conference of Open Innovations Association FRUCT2305-72542343-07372022-11-0132122122610.23919/FRUCT56874.2022.9953838Ontology Concept Extraction Algorithm for Deep Neural NetworksAndrew Ponomarev0Anton Agafonov1SPC RAS, RussiaSPC RAS, RussiaAn important drawback of deep neural networks limiting their application in critical tasks is the lack of explainability. Recently, several methods have been proposed to explain and interpret the results obtained by deep neural networks, however, the majority of these methods are targeted mostly at AI experts. Ontology-based explanation techniques seem promising, as they can be used to form explanations using domain terms (corresponding to ontology concepts) and logical statements, which is more understandable by domain experts. Recently, it has been shown, that inner representations (layer activations) of deep neural network can often be aligned with ontology concepts. However, not every concept can be matched with the output of every layer, and it can be computationally hard to identify the particular layer that can be easily aligned with the given concept, which is aggravated by the number of concepts in a typical ontology. The paper proposes an algorithm to address this problem. For each ontology concept it helps to identify neural network layer, which produces output that can be best aligned with the given concept. These connections can then be used to identify all the ontology concepts relevant to the sample and explain the network output in a user-friendly way.https://www.fruct.org/publications/volume-32/fruct32/files/Pon.pdfexplainable aineural networksneuro-symbolic aiontologiesdeep learning |
| spellingShingle | Andrew Ponomarev Anton Agafonov Ontology Concept Extraction Algorithm for Deep Neural Networks Proceedings of the XXth Conference of Open Innovations Association FRUCT explainable ai neural networks neuro-symbolic ai ontologies deep learning |
| title | Ontology Concept Extraction Algorithm for Deep Neural Networks |
| title_full | Ontology Concept Extraction Algorithm for Deep Neural Networks |
| title_fullStr | Ontology Concept Extraction Algorithm for Deep Neural Networks |
| title_full_unstemmed | Ontology Concept Extraction Algorithm for Deep Neural Networks |
| title_short | Ontology Concept Extraction Algorithm for Deep Neural Networks |
| title_sort | ontology concept extraction algorithm for deep neural networks |
| topic | explainable ai neural networks neuro-symbolic ai ontologies deep learning |
| url | https://www.fruct.org/publications/volume-32/fruct32/files/Pon.pdf |
| work_keys_str_mv | AT andrewponomarev ontologyconceptextractionalgorithmfordeepneuralnetworks AT antonagafonov ontologyconceptextractionalgorithmfordeepneuralnetworks |