Iterative Oblique Decision Trees Deliver Explainable RL Models
The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We in...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/16/6/282 |
| _version_ | 1797596488691351552 |
|---|---|
| author | Raphael C. Engelhardt Marc Oedingen Moritz Lange Laurenz Wiskott Wolfgang Konen |
| author_facet | Raphael C. Engelhardt Marc Oedingen Moritz Lange Laurenz Wiskott Wolfgang Konen |
| author_sort | Raphael C. Engelhardt |
| collection | DOAJ |
| description | The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate three algorithms for generating training data for axis-parallel and oblique DTs with the help of DRL agents (“oracles”) and evaluate these methods on classic control problems from OpenAI Gym. The results show that one of our newly developed algorithms, the iterative training, outperforms traditional sampling algorithms, resulting in well-performing DTs that often even surpass the oracle from which they were trained. Even higher dimensional problems can be solved with surprisingly shallow DTs. We discuss the advantages and disadvantages of different sampling methods and insights into the decision-making process made possible by the transparent nature of DTs. Our work contributes to the development of not only powerful but also explainable RL agents and highlights the potential of DTs as a simple and effective alternative to complex DRL models. |
| first_indexed | 2024-03-11T02:52:06Z |
| format | Article |
| id | doaj.art-c3859472dfec4fe7af6884c5228d2bac |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-11T02:52:06Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-c3859472dfec4fe7af6884c5228d2bac2023-11-18T08:56:45ZengMDPI AGAlgorithms1999-48932023-05-0116628210.3390/a16060282Iterative Oblique Decision Trees Deliver Explainable RL ModelsRaphael C. Engelhardt0Marc Oedingen1Moritz Lange2Laurenz Wiskott3Wolfgang Konen4Cologne Institute of Computer Science, Faculty of Computer Science and Engineering Science, TH Köln, 51643 Gummersbach, GermanyCologne Institute of Computer Science, Faculty of Computer Science and Engineering Science, TH Köln, 51643 Gummersbach, GermanyInstitute for Neural Computation, Faculty of Computer Science, Ruhr-University Bochum, 44801 Bochum, GermanyInstitute for Neural Computation, Faculty of Computer Science, Ruhr-University Bochum, 44801 Bochum, GermanyCologne Institute of Computer Science, Faculty of Computer Science and Engineering Science, TH Köln, 51643 Gummersbach, GermanyThe demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate three algorithms for generating training data for axis-parallel and oblique DTs with the help of DRL agents (“oracles”) and evaluate these methods on classic control problems from OpenAI Gym. The results show that one of our newly developed algorithms, the iterative training, outperforms traditional sampling algorithms, resulting in well-performing DTs that often even surpass the oracle from which they were trained. Even higher dimensional problems can be solved with surprisingly shallow DTs. We discuss the advantages and disadvantages of different sampling methods and insights into the decision-making process made possible by the transparent nature of DTs. Our work contributes to the development of not only powerful but also explainable RL agents and highlights the potential of DTs as a simple and effective alternative to complex DRL models.https://www.mdpi.com/1999-4893/16/6/282reinforcement learningdecision treeexplainable AIrule learning |
| spellingShingle | Raphael C. Engelhardt Marc Oedingen Moritz Lange Laurenz Wiskott Wolfgang Konen Iterative Oblique Decision Trees Deliver Explainable RL Models Algorithms reinforcement learning decision tree explainable AI rule learning |
| title | Iterative Oblique Decision Trees Deliver Explainable RL Models |
| title_full | Iterative Oblique Decision Trees Deliver Explainable RL Models |
| title_fullStr | Iterative Oblique Decision Trees Deliver Explainable RL Models |
| title_full_unstemmed | Iterative Oblique Decision Trees Deliver Explainable RL Models |
| title_short | Iterative Oblique Decision Trees Deliver Explainable RL Models |
| title_sort | iterative oblique decision trees deliver explainable rl models |
| topic | reinforcement learning decision tree explainable AI rule learning |
| url | https://www.mdpi.com/1999-4893/16/6/282 |
| work_keys_str_mv | AT raphaelcengelhardt iterativeobliquedecisiontreesdeliverexplainablerlmodels AT marcoedingen iterativeobliquedecisiontreesdeliverexplainablerlmodels AT moritzlange iterativeobliquedecisiontreesdeliverexplainablerlmodels AT laurenzwiskott iterativeobliquedecisiontreesdeliverexplainablerlmodels AT wolfgangkonen iterativeobliquedecisiontreesdeliverexplainablerlmodels |