MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization.
In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2015-01-01
|
| Series: | PLoS ONE |
| Online Access: | http://europepmc.org/articles/PMC4619362?pdf=render |
| _version_ | 1811294133794373632 |
|---|---|
| author | Olivier Mangin David Filliat Louis Ten Bosch Pierre-Yves Oudeyer |
| author_facet | Olivier Mangin David Filliat Louis Ten Bosch Pierre-Yves Oudeyer |
| author_sort | Olivier Mangin |
| collection | DOAJ |
| description | In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available. |
| first_indexed | 2024-04-13T05:12:09Z |
| format | Article |
| id | doaj.art-79da1e780fec40c1b4b913de062e58f8 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-04-13T05:12:09Z |
| publishDate | 2015-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-79da1e780fec40c1b4b913de062e58f82022-12-22T03:01:00ZengPublic Library of Science (PLoS)PLoS ONE1932-62032015-01-011010e014073210.1371/journal.pone.0140732MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization.Olivier ManginDavid FilliatLouis Ten BoschPierre-Yves OudeyerIn this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available.http://europepmc.org/articles/PMC4619362?pdf=render |
| spellingShingle | Olivier Mangin David Filliat Louis Ten Bosch Pierre-Yves Oudeyer MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. PLoS ONE |
| title | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. |
| title_full | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. |
| title_fullStr | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. |
| title_full_unstemmed | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. |
| title_short | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. |
| title_sort | mca nmf multimodal concept acquisition with non negative matrix factorization |
| url | http://europepmc.org/articles/PMC4619362?pdf=render |
| work_keys_str_mv | AT oliviermangin mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT davidfilliat mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT louistenbosch mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT pierreyvesoudeyer mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization |