Do learning rates adapt to the distribution of rewards?
Studies of reinforcement learning have shown that humans learn differently in response to positive and negative reward prediction errors, a phenomenon that can be captured computationally by positing asymmetric learning rates. This asymmetry, motivated by neurobiological and cognitive considerations...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Springer US
2016
|
| Online Access: | http://hdl.handle.net/1721.1/103813 |
| _version_ | 1826215537184604160 |
|---|---|
| author | Gershman, Samuel J. |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Gershman, Samuel J. |
| author_sort | Gershman, Samuel J. |
| collection | MIT |
| description | Studies of reinforcement learning have shown that humans learn differently in response to positive and negative reward prediction errors, a phenomenon that can be captured computationally by positing asymmetric learning rates. This asymmetry, motivated by neurobiological and cognitive considerations, has been invoked to explain learning differences across the lifespan as well as a range of psychiatric disorders. Recent theoretical work, motivated by normative considerations, has hypothesized that the learning rate asymmetry should be modulated by the distribution of rewards across the available options. In particular, the learning rate for negative prediction errors should be higher than the learning rate for positive prediction errors when the average reward rate is high, and this relationship should reverse when the reward rate is low. We tested this hypothesis in a series of experiments. Contrary to the theoretical predictions, we found that the asymmetry was largely insensitive to the average reward rate; instead, the dominant pattern was a higher learning rate for negative than for positive prediction errors, possibly reflecting risk aversion. |
| first_indexed | 2024-09-23T16:34:03Z |
| format | Article |
| id | mit-1721.1/103813 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T16:34:03Z |
| publishDate | 2016 |
| publisher | Springer US |
| record_format | dspace |
| spelling | mit-1721.1/1038132022-10-02T08:18:17Z Do learning rates adapt to the distribution of rewards? Gershman, Samuel J. Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Gershman, Samuel J. Studies of reinforcement learning have shown that humans learn differently in response to positive and negative reward prediction errors, a phenomenon that can be captured computationally by positing asymmetric learning rates. This asymmetry, motivated by neurobiological and cognitive considerations, has been invoked to explain learning differences across the lifespan as well as a range of psychiatric disorders. Recent theoretical work, motivated by normative considerations, has hypothesized that the learning rate asymmetry should be modulated by the distribution of rewards across the available options. In particular, the learning rate for negative prediction errors should be higher than the learning rate for positive prediction errors when the average reward rate is high, and this relationship should reverse when the reward rate is low. We tested this hypothesis in a series of experiments. Contrary to the theoretical predictions, we found that the asymmetry was largely insensitive to the average reward rate; instead, the dominant pattern was a higher learning rate for negative than for positive prediction errors, possibly reflecting risk aversion. MIT Intelligence Initiative (postdoctoral fellowship) 2016-07-29T20:59:05Z 2016-07-29T20:59:05Z 2015-01 2016-05-23T12:18:09Z Article http://purl.org/eprint/type/JournalArticle 1069-9384 1531-5320 http://hdl.handle.net/1721.1/103813 Gershman, Samuel J. “Do Learning Rates Adapt to the Distribution of Rewards?” Psychonomic Bulletin & Review 22.5 (2015): 1320–1327. en http://dx.doi.org/10.3758/s13423-014-0790-3 Psychonomic Bulletin & Review Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. Psychonomic Society, Inc. application/pdf Springer US Springer US |
| spellingShingle | Gershman, Samuel J. Do learning rates adapt to the distribution of rewards? |
| title | Do learning rates adapt to the distribution of rewards? |
| title_full | Do learning rates adapt to the distribution of rewards? |
| title_fullStr | Do learning rates adapt to the distribution of rewards? |
| title_full_unstemmed | Do learning rates adapt to the distribution of rewards? |
| title_short | Do learning rates adapt to the distribution of rewards? |
| title_sort | do learning rates adapt to the distribution of rewards |
| url | http://hdl.handle.net/1721.1/103813 |
| work_keys_str_mv | AT gershmansamuelj dolearningratesadapttothedistributionofrewards |