Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings.
Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes thr...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2011-01-01
|
| Series: | PLoS Computational Biology |
| Online Access: | https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21283774/pdf/?tool=EBI |
| _version_ | 1818405311860441088 |
|---|---|
| author | Elise Payzan-LeNestour Peter Bossaerts |
| author_facet | Elise Payzan-LeNestour Peter Bossaerts |
| author_sort | Elise Payzan-LeNestour |
| collection | DOAJ |
| description | Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating. |
| first_indexed | 2024-12-14T08:54:02Z |
| format | Article |
| id | doaj.art-288f655dafb4488386e9caabcdee5ab1 |
| institution | Directory Open Access Journal |
| issn | 1553-734X 1553-7358 |
| language | English |
| last_indexed | 2024-12-14T08:54:02Z |
| publishDate | 2011-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Computational Biology |
| spelling | doaj.art-288f655dafb4488386e9caabcdee5ab12022-12-21T23:08:58ZengPublic Library of Science (PLoS)PLoS Computational Biology1553-734X1553-73582011-01-0171e100104810.1371/journal.pcbi.1001048Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings.Elise Payzan-LeNestourPeter BossaertsRecently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21283774/pdf/?tool=EBI |
| spellingShingle | Elise Payzan-LeNestour Peter Bossaerts Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Computational Biology |
| title | Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. |
| title_full | Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. |
| title_fullStr | Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. |
| title_full_unstemmed | Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. |
| title_short | Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. |
| title_sort | risk unexpected uncertainty and estimation uncertainty bayesian learning in unstable settings |
| url | https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21283774/pdf/?tool=EBI |
| work_keys_str_mv | AT elisepayzanlenestour riskunexpecteduncertaintyandestimationuncertaintybayesianlearninginunstablesettings AT peterbossaerts riskunexpecteduncertaintyandestimationuncertaintybayesianlearninginunstablesettings |