CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning
Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. Howeve...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-03-01
|
| Series: | Frontiers in Neuroinformatics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fninf.2023.1134405/full |
| _version_ | 1811157867336564736 |
|---|---|
| author | Nicolas Diekmann Nicolas Diekmann Sandhiya Vijayabaskaran Xiangshuai Zeng Xiangshuai Zeng David Kappel Matheus Chaves Menezes Sen Cheng |
| author_facet | Nicolas Diekmann Nicolas Diekmann Sandhiya Vijayabaskaran Xiangshuai Zeng Xiangshuai Zeng David Kappel Matheus Chaves Menezes Sen Cheng |
| author_sort | Nicolas Diekmann |
| collection | DOAJ |
| description | Reinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience. |
| first_indexed | 2024-04-10T05:13:37Z |
| format | Article |
| id | doaj.art-3fb5c86ff5014178b1b3867676ccbec9 |
| institution | Directory Open Access Journal |
| issn | 1662-5196 |
| language | English |
| last_indexed | 2024-04-10T05:13:37Z |
| publishDate | 2023-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroinformatics |
| spelling | doaj.art-3fb5c86ff5014178b1b3867676ccbec92023-03-09T04:58:16ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962023-03-011710.3389/fninf.2023.11344051134405CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learningNicolas Diekmann0Nicolas Diekmann1Sandhiya Vijayabaskaran2Xiangshuai Zeng3Xiangshuai Zeng4David Kappel5Matheus Chaves Menezes6Sen Cheng7Faculty for Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, GermanyInternational Graduate School of Neuroscience, Ruhr University Bochum, Bochum, GermanyFaculty for Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, GermanyFaculty for Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, GermanyInternational Graduate School of Neuroscience, Ruhr University Bochum, Bochum, GermanyFaculty for Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, GermanyLaboratory of Artificial Cognition Methods for Optimisation and Robotics, Federal University of Maranhão, São Luís, BrazilFaculty for Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, GermanyReinforcement learning (RL) has become a popular paradigm for modeling animal behavior, analyzing neuronal representations, and studying their emergence during learning. This development has been fueled by advances in understanding the role of RL in both the brain and artificial intelligence. However, while in machine learning a set of tools and standardized benchmarks facilitate the development of new methods and their comparison to existing ones, in neuroscience, the software infrastructure is much more fragmented. Even if sharing theoretical principles, computational studies rarely share software frameworks, thereby impeding the integration or comparison of different results. Machine learning tools are also difficult to port to computational neuroscience since the experimental requirements are usually not well aligned. To address these challenges we introduce CoBeL-RL, a closed-loop simulator of complex behavior and learning based on RL and deep neural networks. It provides a neuroscience-oriented framework for efficiently setting up and running simulations. CoBeL-RL offers a set of virtual environments, e.g., T-maze and Morris water maze, which can be simulated at different levels of abstraction, e.g., a simple gridworld or a 3D environment with complex visual stimuli, and set up using intuitive GUI tools. A range of RL algorithms, e.g., Dyna-Q and deep Q-network algorithms, is provided and can be easily extended. CoBeL-RL provides tools for monitoring and analyzing behavior and unit activity, and allows for fine-grained control of the simulation via interfaces to relevant points in its closed-loop. In summary, CoBeL-RL fills an important gap in the software toolbox of computational neuroscience.https://www.frontiersin.org/articles/10.3389/fninf.2023.1134405/fullspatial navigationspatial learninghippocampusplace cellsgrid cellssimulation framework |
| spellingShingle | Nicolas Diekmann Nicolas Diekmann Sandhiya Vijayabaskaran Xiangshuai Zeng Xiangshuai Zeng David Kappel Matheus Chaves Menezes Sen Cheng CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning Frontiers in Neuroinformatics spatial navigation spatial learning hippocampus place cells grid cells simulation framework |
| title | CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning |
| title_full | CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning |
| title_fullStr | CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning |
| title_full_unstemmed | CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning |
| title_short | CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning |
| title_sort | cobel rl a neuroscience oriented simulation framework for complex behavior and learning |
| topic | spatial navigation spatial learning hippocampus place cells grid cells simulation framework |
| url | https://www.frontiersin.org/articles/10.3389/fninf.2023.1134405/full |
| work_keys_str_mv | AT nicolasdiekmann cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT nicolasdiekmann cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT sandhiyavijayabaskaran cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT xiangshuaizeng cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT xiangshuaizeng cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT davidkappel cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT matheuschavesmenezes cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning AT sencheng cobelrlaneuroscienceorientedsimulationframeworkforcomplexbehaviorandlearning |