Counterfactual policy introspection using structural causal models
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/1721.1/124128 |
| _version_ | 1826192935513751552 |
|---|---|
| author | Oberst, Michael Karl. |
| author2 | David Sontag. |
| author_facet | David Sontag. Oberst, Michael Karl. |
| author_sort | Oberst, Michael Karl. |
| collection | MIT |
| description | This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. |
| first_indexed | 2024-09-23T09:31:24Z |
| format | Thesis |
| id | mit-1721.1/124128 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T09:31:24Z |
| publishDate | 2020 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1241282020-10-15T06:34:20Z Counterfactual policy introspection using structural causal models Oberst, Michael Karl. David Sontag. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Electrical Engineering and Computer Science. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 97-102). Inspired by a growing interest in applying reinforcement learning (RL) to healthcare, we introduce a procedure for performing qualitative introspection and `debugging' of models and policies. In particular, we make use of counterfactual trajectories, which describe the implicit belief (of a model) of 'what would have happened' if a policy had been applied. These serve to decompose model-based estimates of reward into specific claims about specific trajectories, a useful tool for 'debugging' of models and policies, especially when side information is available for domain experts to review alongside the counterfactual claims. More specically, we give a general procedure (using structural causal models) to generate counterfactuals based on an existing model of the environment, including common models used in model-based RL. We apply our procedure to a pair of synthetic applications to build intuition, and conclude with an application on real healthcare data, introspecting a policy for sepsis management learned in the recently published work of Komorowski et al. (2018). by Michael Karl Oberst. S.M. S.M. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science 2020-03-09T18:59:12Z 2020-03-09T18:59:12Z 2019 2019 Thesis https://hdl.handle.net/1721.1/124128 1142635604 eng MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided. http://dspace.mit.edu/handle/1721.1/7582 102 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Oberst, Michael Karl. Counterfactual policy introspection using structural causal models |
| title | Counterfactual policy introspection using structural causal models |
| title_full | Counterfactual policy introspection using structural causal models |
| title_fullStr | Counterfactual policy introspection using structural causal models |
| title_full_unstemmed | Counterfactual policy introspection using structural causal models |
| title_short | Counterfactual policy introspection using structural causal models |
| title_sort | counterfactual policy introspection using structural causal models |
| topic | Electrical Engineering and Computer Science. |
| url | https://hdl.handle.net/1721.1/124128 |
| work_keys_str_mv | AT oberstmichaelkarl counterfactualpolicyintrospectionusingstructuralcausalmodels |