Neurorobotic reinforcement learning for domains with parametrical uncertainty
Neuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware i...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-10-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1239581/full |
| _version_ | 1797649462995189760 |
|---|---|
| author | Camilo Amaya Axel von Arnim |
| author_facet | Camilo Amaya Axel von Arnim |
| author_sort | Camilo Amaya |
| collection | DOAJ |
| description | Neuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this study, we used the neurorobotics platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implemented a force-torque feedback-based classic object insertion task (“peg-in-hole”) and controlled the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to real-world parameter variations in the target domain, filling the sim-to-real gap.To the best of our knowledge, it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains. |
| first_indexed | 2024-03-11T15:47:30Z |
| format | Article |
| id | doaj.art-d6e9fe9055d84a799abb1aa0a4ceda4c |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-03-11T15:47:30Z |
| publishDate | 2023-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-d6e9fe9055d84a799abb1aa0a4ceda4c2023-10-26T06:11:54ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182023-10-011710.3389/fnbot.2023.12395811239581Neurorobotic reinforcement learning for domains with parametrical uncertaintyCamilo AmayaAxel von ArnimNeuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this study, we used the neurorobotics platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implemented a force-torque feedback-based classic object insertion task (“peg-in-hole”) and controlled the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to real-world parameter variations in the target domain, filling the sim-to-real gap.To the best of our knowledge, it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains.https://www.frontiersin.org/articles/10.3389/fnbot.2023.1239581/fulldomain randomizationneuromorphic computingneuroroboticsreinforcement learningrobot controlspiking neural networks |
| spellingShingle | Camilo Amaya Axel von Arnim Neurorobotic reinforcement learning for domains with parametrical uncertainty Frontiers in Neurorobotics domain randomization neuromorphic computing neurorobotics reinforcement learning robot control spiking neural networks |
| title | Neurorobotic reinforcement learning for domains with parametrical uncertainty |
| title_full | Neurorobotic reinforcement learning for domains with parametrical uncertainty |
| title_fullStr | Neurorobotic reinforcement learning for domains with parametrical uncertainty |
| title_full_unstemmed | Neurorobotic reinforcement learning for domains with parametrical uncertainty |
| title_short | Neurorobotic reinforcement learning for domains with parametrical uncertainty |
| title_sort | neurorobotic reinforcement learning for domains with parametrical uncertainty |
| topic | domain randomization neuromorphic computing neurorobotics reinforcement learning robot control spiking neural networks |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1239581/full |
| work_keys_str_mv | AT camiloamaya neuroroboticreinforcementlearningfordomainswithparametricaluncertainty AT axelvonarnim neuroroboticreinforcementlearningfordomainswithparametricaluncertainty |