Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting
Humans are known to discount future rewards hyperbolically in time. Nevertheless, a formal recursive model of hyperbolic discounting has been elusive until recently, with the introduction of the hyperbolically discounted temporal difference (HDTD) model. Prior to that, models of learning (especial...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2014-03-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fpsyg.2014.00178/full |
| _version_ | 1818108798155358208 |
|---|---|
| author | Noah eZarr Joshua W. Brown William eAlexander |
| author_facet | Noah eZarr Joshua W. Brown William eAlexander |
| author_sort | Noah eZarr |
| collection | DOAJ |
| description | Humans are known to discount future rewards hyperbolically in time. Nevertheless, a formal recursive model of hyperbolic discounting has been elusive until recently, with the introduction of the hyperbolically discounted temporal difference (HDTD) model. Prior to that, models of learning (especially reinforcement learning) have relied on exponential discounting, which generally provides poorer fits to behavioral data. Recently, it has been shown that hyperbolic discounting can also be approximated by a summed distribution of exponentially discounted values, instantiated in the µAgents model. The HDTD model and the µAgents model differ in one key respect, namely how they treat sequences of rewards. The µAgents model is a particular implementation of a parallel discounting model, which values sequences based on the summed value of the individual rewards whereas the HDTD model contains a nonlinear interaction. To discriminate among these models, we ascertained how subjects discounted a sequence of three rewards, and then we tested how well each candidate model fit the subject data. The results show that the parallel model generally provides a better fit to the human data. |
| first_indexed | 2024-12-11T02:21:05Z |
| format | Article |
| id | doaj.art-6907623bdcb74b81bde77c94b0caafaf |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-11T02:21:05Z |
| publishDate | 2014-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-6907623bdcb74b81bde77c94b0caafaf2022-12-22T01:24:03ZengFrontiers Media S.A.Frontiers in Psychology1664-10782014-03-01510.3389/fpsyg.2014.0017871173Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic DiscountingNoah eZarr0Joshua W. Brown1William eAlexander2Indiana UniversityIndiana UniversityGhent UniversityHumans are known to discount future rewards hyperbolically in time. Nevertheless, a formal recursive model of hyperbolic discounting has been elusive until recently, with the introduction of the hyperbolically discounted temporal difference (HDTD) model. Prior to that, models of learning (especially reinforcement learning) have relied on exponential discounting, which generally provides poorer fits to behavioral data. Recently, it has been shown that hyperbolic discounting can also be approximated by a summed distribution of exponentially discounted values, instantiated in the µAgents model. The HDTD model and the µAgents model differ in one key respect, namely how they treat sequences of rewards. The µAgents model is a particular implementation of a parallel discounting model, which values sequences based on the summed value of the individual rewards whereas the HDTD model contains a nonlinear interaction. To discriminate among these models, we ascertained how subjects discounted a sequence of three rewards, and then we tested how well each candidate model fit the subject data. The results show that the parallel model generally provides a better fit to the human data.http://journal.frontiersin.org/Journal/10.3389/fpsyg.2014.00178/fullBehavioral Researchdiscountingmodel fittingExponential discountinghyperbolic discountingtemporal difference learning |
| spellingShingle | Noah eZarr Joshua W. Brown William eAlexander Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting Frontiers in Psychology Behavioral Research discounting model fitting Exponential discounting hyperbolic discounting temporal difference learning |
| title | Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting |
| title_full | Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting |
| title_fullStr | Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting |
| title_full_unstemmed | Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting |
| title_short | Discounting of Reward Sequences: a Test of Competing Formal Models of Hyperbolic Discounting |
| title_sort | discounting of reward sequences a test of competing formal models of hyperbolic discounting |
| topic | Behavioral Research discounting model fitting Exponential discounting hyperbolic discounting temporal difference learning |
| url | http://journal.frontiersin.org/Journal/10.3389/fpsyg.2014.00178/full |
| work_keys_str_mv | AT noahezarr discountingofrewardsequencesatestofcompetingformalmodelsofhyperbolicdiscounting AT joshuawbrown discountingofrewardsequencesatestofcompetingformalmodelsofhyperbolicdiscounting AT williamealexander discountingofrewardsequencesatestofcompetingformalmodelsofhyperbolicdiscounting |