Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions
Interactive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss con...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Machine Learning and Knowledge Extraction |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-4990/4/4/50 |
| _version_ | 1797456623972646912 |
|---|---|
| author | Sebastian Kiefer Mareike Hoffmann Ute Schmid |
| author_facet | Sebastian Kiefer Mareike Hoffmann Ute Schmid |
| author_sort | Sebastian Kiefer |
| collection | DOAJ |
| description | Interactive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies such as CAIPI are limited to ’destructive’ feedback, meaning that they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called <i>Semantic Interactive Learning</i> for the domain of document classification, located at the intersection between Natural Language Processing (NLP) and Machine Learning (ML). We frame the problem of incorporating constructive and contextual feedback into the learner as a task involving finding an architecture that enables more semantic alignment between humans and machines while at the same time helping to maintain the statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner’s reasoning is pushed towards the desired behavior. Through several experiments we show how our method compares to CAIPI, a state of the art IML strategy, in terms of Predictive Performance and Local Explanation Quality in downstream multi-class classification tasks. Especially in the early stages of interactions, our proposed method clearly outperforms CAIPI while allowing for contextual interpretation and intervention. Overall, SemanticPush stands out with regard to data efficiency, as it requires fewer queries from the pool dataset to achieve high accuracy. |
| first_indexed | 2024-03-09T16:10:30Z |
| format | Article |
| id | doaj.art-e74222e2ec6a4e6bb47dba82ec424428 |
| institution | Directory Open Access Journal |
| issn | 2504-4990 |
| language | English |
| last_indexed | 2024-03-09T16:10:30Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machine Learning and Knowledge Extraction |
| spelling | doaj.art-e74222e2ec6a4e6bb47dba82ec4244282023-11-24T16:19:01ZengMDPI AGMachine Learning and Knowledge Extraction2504-49902022-11-0144994101010.3390/make4040050Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual InteractionsSebastian Kiefer0Mareike Hoffmann1Ute Schmid2Cognitive Systems, University of Bamberg, 96047 Bamberg, GermanyCognitive Systems, University of Bamberg, 96047 Bamberg, GermanyCognitive Systems, University of Bamberg, 96047 Bamberg, GermanyInteractive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies such as CAIPI are limited to ’destructive’ feedback, meaning that they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called <i>Semantic Interactive Learning</i> for the domain of document classification, located at the intersection between Natural Language Processing (NLP) and Machine Learning (ML). We frame the problem of incorporating constructive and contextual feedback into the learner as a task involving finding an architecture that enables more semantic alignment between humans and machines while at the same time helping to maintain the statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner’s reasoning is pushed towards the desired behavior. Through several experiments we show how our method compares to CAIPI, a state of the art IML strategy, in terms of Predictive Performance and Local Explanation Quality in downstream multi-class classification tasks. Especially in the early stages of interactions, our proposed method clearly outperforms CAIPI while allowing for contextual interpretation and intervention. Overall, SemanticPush stands out with regard to data efficiency, as it requires fewer queries from the pool dataset to achieve high accuracy.https://www.mdpi.com/2504-4990/4/4/50human-centric machine learninginteractive machine learningCAIPIexplainable artificial intelligencelocal surrogate explanation modelscontextual and semantic explanations |
| spellingShingle | Sebastian Kiefer Mareike Hoffmann Ute Schmid Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions Machine Learning and Knowledge Extraction human-centric machine learning interactive machine learning CAIPI explainable artificial intelligence local surrogate explanation models contextual and semantic explanations |
| title | Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions |
| title_full | Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions |
| title_fullStr | Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions |
| title_full_unstemmed | Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions |
| title_short | Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions |
| title_sort | semantic interactive learning for text classification a constructive approach for contextual interactions |
| topic | human-centric machine learning interactive machine learning CAIPI explainable artificial intelligence local surrogate explanation models contextual and semantic explanations |
| url | https://www.mdpi.com/2504-4990/4/4/50 |
| work_keys_str_mv | AT sebastiankiefer semanticinteractivelearningfortextclassificationaconstructiveapproachforcontextualinteractions AT mareikehoffmann semanticinteractivelearningfortextclassificationaconstructiveapproachforcontextualinteractions AT uteschmid semanticinteractivelearningfortextclassificationaconstructiveapproachforcontextualinteractions |