Compositional simulation in perception and cognition
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2019
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/1721.1/121814 |
| _version_ | 1826215401459023872 |
|---|---|
| author | Siegel, Max Harmon. |
| author2 | Joshua B. Tenenbaum. |
| author_facet | Joshua B. Tenenbaum. Siegel, Max Harmon. |
| author_sort | Siegel, Max Harmon. |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2019 |
| first_indexed | 2024-09-23T16:27:24Z |
| format | Thesis |
| id | mit-1721.1/121814 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T16:27:24Z |
| publishDate | 2019 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1218142019-09-13T03:02:45Z Compositional simulation in perception and cognition Siegel, Max Harmon. Joshua B. Tenenbaum. Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences. Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Brain and Cognitive Sciences. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2019 Cataloged from PDF version of thesis. "February 2019." Includes bibliographical references (pages 97-103). Despite rapid recent progress in machine perception and models of biological perception, fundamental questions remain open. In particular, the paradigm underlying these advances, pattern recognition, requires large amounts of training data and struggles to generalize to situations outside the domain of training. In this thesis, I focus on a broad class of perceptual concepts - those that are generated by the composition of multiple causal processes, in this case certain physical interactions - that human use essentially and effortlessly in making sense of the world, but for which any specific instance is extremely rare in our experience. Pattern recognition, or any strongly learning-based approach, might then be an inappropriate way to understand people's perceptual inferences. I propose an alternative approach, compositional simulation, that can in principle account for these inferences, and I show in practice that it provides both qualitative and quantitative explanatory value for several experimental settings. Consider a box and a number of marbles in the box, and imagine trying to guess how many there are based on the sound produced when the box is shaken. I demonstrate that human observers are quite good at this task, even for subtle numerical differences. Compositional simulation hypothesizes that people succeed by leveraging internal causal models: they simulate the physical collisions that would result from shaking the box (in a particular way), and what those collisions would sound like, for different numbers of marbles. They then compare their simulated sounds with the sound they heard. Crucially these simulation models can generalize to a wide range of percepts, even those never before experienced, by exploiting the compositional structure of the causal processes being modeled, in terms of objects and their interactions, and physical dynamics and auditory events. Because the motion of the box is a key ingredient in physical simulation, I hypothesize that people can take cues to motion into account in our task; I give evidence that people do. I also consider the domain of unfamiliar objects covered by cloth. a similar mechanism should enable successful recognition even for unfamiliar covered objects (like airplanes). I show that people can succeed in the recognition task, even when the shape of the object is very different when covered. Finally, I show how compositional simulation provides a way to "glue together" the data received by perception (images and sounds) with the contents of cognition (objects). I apply compositional simulation to two cognitive domains: children's intuitive exploration (obtaining quantitative prediction of exploration time), and causal inference from audiovisual information. by Max Harmon Siegel. Ph. D. Ph.D. Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences 2019-07-18T20:32:09Z 2019-07-18T20:32:09Z 2018 2019 Thesis https://hdl.handle.net/1721.1/121814 1103712575 eng MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 103 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Brain and Cognitive Sciences. Siegel, Max Harmon. Compositional simulation in perception and cognition |
| title | Compositional simulation in perception and cognition |
| title_full | Compositional simulation in perception and cognition |
| title_fullStr | Compositional simulation in perception and cognition |
| title_full_unstemmed | Compositional simulation in perception and cognition |
| title_short | Compositional simulation in perception and cognition |
| title_sort | compositional simulation in perception and cognition |
| topic | Brain and Cognitive Sciences. |
| url | https://hdl.handle.net/1721.1/121814 |
| work_keys_str_mv | AT siegelmaxharmon compositionalsimulationinperceptionandcognition |