XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
The lack of transparency of powerful Machine Learning systems paired with their growth in popularity over the last decade led to the emergence of the eXplainable Artificial Intelligence (XAI) field. Instead of focusing solely on obtaining highly performing models, researchers also develop explanatio...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/19/9423 |
| _version_ | 1797480875439423488 |
|---|---|
| author | Pedro Lopes Eduardo Silva Cristiana Braga Tiago Oliveira Luís Rosado |
| author_facet | Pedro Lopes Eduardo Silva Cristiana Braga Tiago Oliveira Luís Rosado |
| author_sort | Pedro Lopes |
| collection | DOAJ |
| description | The lack of transparency of powerful Machine Learning systems paired with their growth in popularity over the last decade led to the emergence of the eXplainable Artificial Intelligence (XAI) field. Instead of focusing solely on obtaining highly performing models, researchers also develop explanation techniques that help better understand the system’s reasoning for a particular output. An explainable system can be designed, developed, and evaluated from different perspectives, which enables researchers from different disciplines to work together on this topic. However, the multidisciplinary nature of XAI systems creates new challenges for condensing and structuring adequate methodologies to design and evaluate such systems. This paper presents a survey of Human-centred and Computer-centred methods to evaluate XAI systems. We propose a new taxonomy to categorize XAI evaluation methods more clearly and intuitively. This categorization gathers knowledge from different disciplines and organizes the evaluation methods according to a set of categories that represent key properties of XAI systems. Possible ways to use the proposed taxonomy in the design and evaluation of XAI systems are also discussed, alongside with some concluding remarks and future directions of research. |
| first_indexed | 2024-03-09T22:07:22Z |
| format | Article |
| id | doaj.art-59ab4ad27139412ca90b6de1a1998314 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T22:07:22Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-59ab4ad27139412ca90b6de1a19983142023-11-23T19:39:13ZengMDPI AGApplied Sciences2076-34172022-09-011219942310.3390/app12199423XAI Systems Evaluation: A Review of Human and Computer-Centred MethodsPedro Lopes0Eduardo Silva1Cristiana Braga2Tiago Oliveira3Luís Rosado4Fraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, PortugalFraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, PortugalFraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, PortugalFirst Solutions—Sistemas de Informação S.A., 4450-102 Matosinhos, PortugalFraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, PortugalThe lack of transparency of powerful Machine Learning systems paired with their growth in popularity over the last decade led to the emergence of the eXplainable Artificial Intelligence (XAI) field. Instead of focusing solely on obtaining highly performing models, researchers also develop explanation techniques that help better understand the system’s reasoning for a particular output. An explainable system can be designed, developed, and evaluated from different perspectives, which enables researchers from different disciplines to work together on this topic. However, the multidisciplinary nature of XAI systems creates new challenges for condensing and structuring adequate methodologies to design and evaluate such systems. This paper presents a survey of Human-centred and Computer-centred methods to evaluate XAI systems. We propose a new taxonomy to categorize XAI evaluation methods more clearly and intuitively. This categorization gathers knowledge from different disciplines and organizes the evaluation methods according to a set of categories that represent key properties of XAI systems. Possible ways to use the proposed taxonomy in the design and evaluation of XAI systems are also discussed, alongside with some concluding remarks and future directions of research.https://www.mdpi.com/2076-3417/12/19/9423explainable artificial intelligenceevaluation methodshuman-centredcomputer-centredliterature review |
| spellingShingle | Pedro Lopes Eduardo Silva Cristiana Braga Tiago Oliveira Luís Rosado XAI Systems Evaluation: A Review of Human and Computer-Centred Methods Applied Sciences explainable artificial intelligence evaluation methods human-centred computer-centred literature review |
| title | XAI Systems Evaluation: A Review of Human and Computer-Centred Methods |
| title_full | XAI Systems Evaluation: A Review of Human and Computer-Centred Methods |
| title_fullStr | XAI Systems Evaluation: A Review of Human and Computer-Centred Methods |
| title_full_unstemmed | XAI Systems Evaluation: A Review of Human and Computer-Centred Methods |
| title_short | XAI Systems Evaluation: A Review of Human and Computer-Centred Methods |
| title_sort | xai systems evaluation a review of human and computer centred methods |
| topic | explainable artificial intelligence evaluation methods human-centred computer-centred literature review |
| url | https://www.mdpi.com/2076-3417/12/19/9423 |
| work_keys_str_mv | AT pedrolopes xaisystemsevaluationareviewofhumanandcomputercentredmethods AT eduardosilva xaisystemsevaluationareviewofhumanandcomputercentredmethods AT cristianabraga xaisystemsevaluationareviewofhumanandcomputercentredmethods AT tiagooliveira xaisystemsevaluationareviewofhumanandcomputercentredmethods AT luisrosado xaisystemsevaluationareviewofhumanandcomputercentredmethods |