Mixtures of strategies underlie rodent behavior during reversal learning.
In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-09-01
|
| Series: | PLoS Computational Biology |
| Online Access: | https://doi.org/10.1371/journal.pcbi.1011430 |
| _version_ | 1797674081044135936 |
|---|---|
| author | Nhat Minh Le Murat Yildirim Yizhi Wang Hiroki Sugihara Mehrdad Jazayeri Mriganka Sur |
| author_facet | Nhat Minh Le Murat Yildirim Yizhi Wang Hiroki Sugihara Mehrdad Jazayeri Mriganka Sur |
| author_sort | Nhat Minh Le |
| collection | DOAJ |
| description | In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions. |
| first_indexed | 2024-03-11T21:54:57Z |
| format | Article |
| id | doaj.art-a9adafe9053d40a891d4518235310437 |
| institution | Directory Open Access Journal |
| issn | 1553-734X 1553-7358 |
| language | English |
| last_indexed | 2024-03-11T21:54:57Z |
| publishDate | 2023-09-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Computational Biology |
| spelling | doaj.art-a9adafe9053d40a891d45182353104372023-09-26T05:30:46ZengPublic Library of Science (PLoS)PLoS Computational Biology1553-734X1553-73582023-09-01199e101143010.1371/journal.pcbi.1011430Mixtures of strategies underlie rodent behavior during reversal learning.Nhat Minh LeMurat YildirimYizhi WangHiroki SugiharaMehrdad JazayeriMriganka SurIn reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.https://doi.org/10.1371/journal.pcbi.1011430 |
| spellingShingle | Nhat Minh Le Murat Yildirim Yizhi Wang Hiroki Sugihara Mehrdad Jazayeri Mriganka Sur Mixtures of strategies underlie rodent behavior during reversal learning. PLoS Computational Biology |
| title | Mixtures of strategies underlie rodent behavior during reversal learning. |
| title_full | Mixtures of strategies underlie rodent behavior during reversal learning. |
| title_fullStr | Mixtures of strategies underlie rodent behavior during reversal learning. |
| title_full_unstemmed | Mixtures of strategies underlie rodent behavior during reversal learning. |
| title_short | Mixtures of strategies underlie rodent behavior during reversal learning. |
| title_sort | mixtures of strategies underlie rodent behavior during reversal learning |
| url | https://doi.org/10.1371/journal.pcbi.1011430 |
| work_keys_str_mv | AT nhatminhle mixturesofstrategiesunderlierodentbehaviorduringreversallearning AT muratyildirim mixturesofstrategiesunderlierodentbehaviorduringreversallearning AT yizhiwang mixturesofstrategiesunderlierodentbehaviorduringreversallearning AT hirokisugihara mixturesofstrategiesunderlierodentbehaviorduringreversallearning AT mehrdadjazayeri mixturesofstrategiesunderlierodentbehaviorduringreversallearning AT mrigankasur mixturesofstrategiesunderlierodentbehaviorduringreversallearning |