Multimodal dynamics : self-supervised learning in perceptual and motor systems
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.
| 主要作者: | |
|---|---|
| 其他作者: | |
| 格式: | Thesis |
| 语言: | eng |
| 出版: |
Massachusetts Institute of Technology
2006
|
| 主题: | |
| 在线阅读: | http://hdl.handle.net/1721.1/34022 |
| _version_ | 1826212419134816256 |
|---|---|
| author | Coen, Michael Harlan |
| author2 | Whitman Richards and Howard Shrobe. |
| author_facet | Whitman Richards and Howard Shrobe. Coen, Michael Harlan |
| author_sort | Coen, Michael Harlan |
| collection | MIT |
| description | Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006. |
| first_indexed | 2024-09-23T15:21:22Z |
| format | Thesis |
| id | mit-1721.1/34022 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T15:21:22Z |
| publishDate | 2006 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/340222019-04-10T08:06:12Z Multimodal dynamics : self-supervised learning in perceptual and motor systems Coen, Michael Harlan Whitman Richards and Howard Shrobe. Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science Electrical Engineering and Computer Science Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Includes bibliographical references (leaves 178-192). This thesis presents a self-supervised framework for perceptual and motor learning based upon correlations in different sensory modalities. The brain and cognitive sciences have gathered an enormous body of neurological and phenomenological evidence in the past half century demonstrating the extraordinary degree of interaction between sensory modalities during the course of ordinary perception. We develop a framework for creating artificial perceptual systems that draws on these findings, where the primary architectural motif is the cross-modal transmission of perceptual information to enhance each sensory channel individually. We present self-supervised algorithms for learning perceptual grounding, intersensory influence, and sensorymotor coordination, which derive training signals from internal cross-modal correlations rather than from external supervision. Our goal is to create systems that develop by interacting with the world around them, inspired by development in animals. We demonstrate this framework with: (1) a system that learns the number and structure of vowels in American English by simultaneously watching and listening to someone speak. The system then cross-modally clusters the correlated auditory and visual data. (cont.) It has no advance linguistic knowledge and receives no information outside of its sensory channels. This work is the first unsupervised acquisition of phonetic structure of which we are aware, outside of that done by human infants. (2) a system that learns to sing like a zebra finch, following the developmental stages of a juvenile zebra finch. It first learns the song of an adult male and then listens to its own initially nascent attempts at mimicry through an articulatory synthesizer. In acquiring the birdsong to which it was initially exposed, this system demonstrates self-supervised sensorimotor learning. It also demonstrates afferent and efferent equivalence - the system learns motor maps with the same computational framework used for learning sensory maps. by Michael Harlan Coen. Ph.D. 2006-09-28T14:51:29Z 2006-09-28T14:51:29Z 2006 2006 Thesis http://hdl.handle.net/1721.1/34022 71315358 eng M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 193 leaves 13746497 bytes 13746265 bytes application/pdf application/pdf application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science Coen, Michael Harlan Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title | Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title_full | Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title_fullStr | Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title_full_unstemmed | Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title_short | Multimodal dynamics : self-supervised learning in perceptual and motor systems |
| title_sort | multimodal dynamics self supervised learning in perceptual and motor systems |
| topic | Electrical Engineering and Computer Science |
| url | http://hdl.handle.net/1721.1/34022 |
| work_keys_str_mv | AT coenmichaelharlan multimodaldynamicsselfsupervisedlearninginperceptualandmotorsystems |