Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models
IntroductionAssociating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an importan...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-01-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2024.1330512/full |
| _version_ | 1827380426630496256 |
|---|---|
| author | Kamma Noda Takafumi Soda Yuichi Yamashita |
| author_facet | Kamma Noda Takafumi Soda Yuichi Yamashita |
| author_sort | Kamma Noda |
| collection | DOAJ |
| description | IntroductionAssociating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills.MethodsWe employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks.ResultsMultimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks.DiscussionOur novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions. |
| first_indexed | 2024-03-08T13:33:37Z |
| format | Article |
| id | doaj.art-aa4156f775f54b4fb5f2107ebf244408 |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-03-08T13:33:37Z |
| publishDate | 2024-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-aa4156f775f54b4fb5f2107ebf2444082024-01-17T04:24:00ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2024-01-011810.3389/fnins.2024.13305121330512Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network modelsKamma NodaTakafumi SodaYuichi YamashitaIntroductionAssociating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills.MethodsWe employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks.ResultsMultimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks.DiscussionOur novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions.https://www.frontiersin.org/articles/10.3389/fnins.2024.1330512/fulldeep learningrepresentation learningmultimodal learningsensory integrationnumerositymathematical ability |
| spellingShingle | Kamma Noda Takafumi Soda Yuichi Yamashita Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models Frontiers in Neuroscience deep learning representation learning multimodal learning sensory integration numerosity mathematical ability |
| title | Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models |
| title_full | Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models |
| title_fullStr | Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models |
| title_full_unstemmed | Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models |
| title_short | Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models |
| title_sort | emergence of number sense through the integration of multimodal information developmental learning insights from neural network models |
| topic | deep learning representation learning multimodal learning sensory integration numerosity mathematical ability |
| url | https://www.frontiersin.org/articles/10.3389/fnins.2024.1330512/full |
| work_keys_str_mv | AT kammanoda emergenceofnumbersensethroughtheintegrationofmultimodalinformationdevelopmentallearninginsightsfromneuralnetworkmodels AT takafumisoda emergenceofnumbersensethroughtheintegrationofmultimodalinformationdevelopmentallearninginsightsfromneuralnetworkmodels AT yuichiyamashita emergenceofnumbersensethroughtheintegrationofmultimodalinformationdevelopmentallearninginsightsfromneuralnetworkmodels |