Augmenting Transformers for Open Domain Procedural Text Comprehesion
Recent advances in deep learning model architectures have permitted state-of-the-art results in various fields such as NLP and CV. Although these systems have matched and, in some cases, surpassed human performance, many of them are still treated as black boxes, with sometimes unpredictable results....
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/139980 |
| _version_ | 1826196575695667200 |
|---|---|
| author | Pei, Yixuan |
| author2 | Shrobe, Howard |
| author_facet | Shrobe, Howard Pei, Yixuan |
| author_sort | Pei, Yixuan |
| collection | MIT |
| description | Recent advances in deep learning model architectures have permitted state-of-the-art results in various fields such as NLP and CV. Although these systems have matched and, in some cases, surpassed human performance, many of them are still treated as black boxes, with sometimes unpredictable results. To try and shed some light on behaviors of natural language generation models, we examine the task of procedural text comprehension using neuro-symbolic techniques. We use this task as a testbed for exploring the limitations of state-of-the-art systems such as GPT on the task of predicting the resulting state changes from the text description of a procedure. We also experiment with whether and how symbolic augmentations may help these systems with understanding language. We see some promising results in concept-net knowledge injection, and note that other augmentations provide more natural generations. |
| first_indexed | 2024-09-23T10:29:55Z |
| format | Thesis |
| id | mit-1721.1/139980 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:29:55Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1399802022-02-08T03:49:52Z Augmenting Transformers for Open Domain Procedural Text Comprehesion Pei, Yixuan Shrobe, Howard Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Recent advances in deep learning model architectures have permitted state-of-the-art results in various fields such as NLP and CV. Although these systems have matched and, in some cases, surpassed human performance, many of them are still treated as black boxes, with sometimes unpredictable results. To try and shed some light on behaviors of natural language generation models, we examine the task of procedural text comprehension using neuro-symbolic techniques. We use this task as a testbed for exploring the limitations of state-of-the-art systems such as GPT on the task of predicting the resulting state changes from the text description of a procedure. We also experiment with whether and how symbolic augmentations may help these systems with understanding language. We see some promising results in concept-net knowledge injection, and note that other augmentations provide more natural generations. M.Eng. 2022-02-07T15:16:54Z 2022-02-07T15:16:54Z 2021-09 2021-11-03T19:25:25.983Z Thesis https://hdl.handle.net/1721.1/139980 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Pei, Yixuan Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title | Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title_full | Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title_fullStr | Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title_full_unstemmed | Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title_short | Augmenting Transformers for Open Domain Procedural Text Comprehesion |
| title_sort | augmenting transformers for open domain procedural text comprehesion |
| url | https://hdl.handle.net/1721.1/139980 |
| work_keys_str_mv | AT peiyixuan augmentingtransformersforopendomainproceduraltextcomprehesion |