Contrasting action and posture coding with hierarchical deep neural network models of proprioception
Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body’s state. The canonical role of proprioception is representing the bo...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
eLife Sciences Publications Ltd
2023-05-01
|
| Series: | eLife |
| Subjects: | |
| Online Access: | https://elifesciences.org/articles/81499 |
| _version_ | 1797776068545871872 |
|---|---|
| author | Kai J Sandbrink Pranav Mamidanna Claudio Michaelis Matthias Bethge Mackenzie Weygandt Mathis Alexander Mathis |
| author_facet | Kai J Sandbrink Pranav Mamidanna Claudio Michaelis Matthias Bethge Mackenzie Weygandt Mathis Alexander Mathis |
| author_sort | Kai J Sandbrink |
| collection | DOAJ |
| description | Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body’s state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one’s posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks’ units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control. |
| first_indexed | 2024-03-12T22:44:41Z |
| format | Article |
| id | doaj.art-e3eef756bddc4fdd9b745407ca1fc4d9 |
| institution | Directory Open Access Journal |
| issn | 2050-084X |
| language | English |
| last_indexed | 2024-03-12T22:44:41Z |
| publishDate | 2023-05-01 |
| publisher | eLife Sciences Publications Ltd |
| record_format | Article |
| series | eLife |
| spelling | doaj.art-e3eef756bddc4fdd9b745407ca1fc4d92023-07-21T08:25:29ZengeLife Sciences Publications LtdeLife2050-084X2023-05-011210.7554/eLife.81499Contrasting action and posture coding with hierarchical deep neural network models of proprioceptionKai J Sandbrink0Pranav Mamidanna1https://orcid.org/0000-0002-2095-3314Claudio Michaelis2Matthias Bethge3https://orcid.org/0000-0002-6417-7812Mackenzie Weygandt Mathis4https://orcid.org/0000-0001-7368-4456Alexander Mathis5https://orcid.org/0000-0002-3777-2202The Rowland Institute at Harvard, Harvard University, Cambridge, United StatesTübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical Physics, Tübingen, GermanyTübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical Physics, Tübingen, GermanyTübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical Physics, Tübingen, GermanyThe Rowland Institute at Harvard, Harvard University, Cambridge, United States; Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Genève, SwitzerlandThe Rowland Institute at Harvard, Harvard University, Cambridge, United States; Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Genève, SwitzerlandBiological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body’s state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one’s posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks’ units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.https://elifesciences.org/articles/81499proprioceptionsomatosensorydeep learningbiomechanicstask-driven modelingsensory systems |
| spellingShingle | Kai J Sandbrink Pranav Mamidanna Claudio Michaelis Matthias Bethge Mackenzie Weygandt Mathis Alexander Mathis Contrasting action and posture coding with hierarchical deep neural network models of proprioception eLife proprioception somatosensory deep learning biomechanics task-driven modeling sensory systems |
| title | Contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| title_full | Contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| title_fullStr | Contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| title_full_unstemmed | Contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| title_short | Contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| title_sort | contrasting action and posture coding with hierarchical deep neural network models of proprioception |
| topic | proprioception somatosensory deep learning biomechanics task-driven modeling sensory systems |
| url | https://elifesciences.org/articles/81499 |
| work_keys_str_mv | AT kaijsandbrink contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception AT pranavmamidanna contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception AT claudiomichaelis contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception AT matthiasbethge contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception AT mackenzieweygandtmathis contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception AT alexandermathis contrastingactionandposturecodingwithhierarchicaldeepneuralnetworkmodelsofproprioception |