Learning, Reasoning, and Planning with Relational and Temporal Neural Networks
Every day, people interpret events and actions in terms of concepts, defined over evolving relations among agents and objects and their goals. We learn these concepts from a limited amount of data, generalizing directly over different numbers and arrangements of agents and objects, and detailed timi...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/139905 |
| _version_ | 1826203799079878656 |
|---|---|
| author | Mao, Jiayuan |
| author2 | Kaelbling, Leslie Pack |
| author_facet | Kaelbling, Leslie Pack Mao, Jiayuan |
| author_sort | Mao, Jiayuan |
| collection | MIT |
| description | Every day, people interpret events and actions in terms of concepts, defined over evolving relations among agents and objects and their goals. We learn these concepts from a limited amount of data, generalizing directly over different numbers and arrangements of agents and objects, and detailed timings of trajectories. We also effectively recompose these concepts to describe unseen behaviors from other agents, and leverage the causal relationships among actions to make plans for ourselves.
This thesis gives an overview of a neuro-symbolic framework for learning, reasoning, and planning with relational and temporal neural networks. The key idea is to exploit a structural bias in neural network learning that enables us to describe complex relational-temporal events and actions. These structures form a minimal amount of prior knowledge but are generic and crucial: scenes are composed of objects; events are temporally related; actions have preconditions and goals. Our systems learn from trajectories with rich temporal and relational patterns and labels for events and actions. We demonstrate that they can generalize from small amounts of data to scenarios containing more objects than were present during training and to temporal warpings of input sequences, and exploits the goal-centric representation of actions to make plans for novel goals. |
| first_indexed | 2024-09-23T12:43:13Z |
| format | Thesis |
| id | mit-1721.1/139905 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T12:43:13Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1399052022-02-08T03:35:00Z Learning, Reasoning, and Planning with Relational and Temporal Neural Networks Mao, Jiayuan Kaelbling, Leslie Pack Tenenbaum, Joshua B. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Every day, people interpret events and actions in terms of concepts, defined over evolving relations among agents and objects and their goals. We learn these concepts from a limited amount of data, generalizing directly over different numbers and arrangements of agents and objects, and detailed timings of trajectories. We also effectively recompose these concepts to describe unseen behaviors from other agents, and leverage the causal relationships among actions to make plans for ourselves. This thesis gives an overview of a neuro-symbolic framework for learning, reasoning, and planning with relational and temporal neural networks. The key idea is to exploit a structural bias in neural network learning that enables us to describe complex relational-temporal events and actions. These structures form a minimal amount of prior knowledge but are generic and crucial: scenes are composed of objects; events are temporally related; actions have preconditions and goals. Our systems learn from trajectories with rich temporal and relational patterns and labels for events and actions. We demonstrate that they can generalize from small amounts of data to scenarios containing more objects than were present during training and to temporal warpings of input sequences, and exploits the goal-centric representation of actions to make plans for novel goals. S.M. 2022-02-07T15:11:51Z 2022-02-07T15:11:51Z 2021-09 2021-09-21T19:54:14.521Z Thesis https://hdl.handle.net/1721.1/139905 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Mao, Jiayuan Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title | Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title_full | Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title_fullStr | Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title_full_unstemmed | Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title_short | Learning, Reasoning, and Planning with Relational and Temporal Neural Networks |
| title_sort | learning reasoning and planning with relational and temporal neural networks |
| url | https://hdl.handle.net/1721.1/139905 |
| work_keys_str_mv | AT maojiayuan learningreasoningandplanningwithrelationalandtemporalneuralnetworks |