The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control
The sex hormone estradiol has recently gained attention in human decision-making research. Animal studies have already shown that estradiol promotes dopaminergic transmission and thus supports reward-seeking behavior and aspects of addiction. In humans, natural variations of estradiol across the men...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-07-01
|
| Series: | Frontiers in Behavioral Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbeh.2021.658769/full |
| _version_ | 1818846817378369536 |
|---|---|
| author | Esther K. Diekhof Andra Geana Andra Geana Frederike Ohm Bradley B. Doll Bradley B. Doll Michael J. Frank Michael J. Frank |
| author_facet | Esther K. Diekhof Andra Geana Andra Geana Frederike Ohm Bradley B. Doll Bradley B. Doll Michael J. Frank Michael J. Frank |
| author_sort | Esther K. Diekhof |
| collection | DOAJ |
| description | The sex hormone estradiol has recently gained attention in human decision-making research. Animal studies have already shown that estradiol promotes dopaminergic transmission and thus supports reward-seeking behavior and aspects of addiction. In humans, natural variations of estradiol across the menstrual cycle modulate the ability to learn from direct performance feedback (“model-free” learning). However, it remains unclear whether estradiol also influences more complex “model-based” contributions to reinforcement learning. Here, 41 women were tested twice – in the low and high estradiol state of the follicular phase of their menstrual cycle – with a Two-Step decision task designed to separate model-free from model-based learning. The results showed that in the high estradiol state women relied more heavily on model-free learning, and accomplished reduced performance gains, particularly during the more volatile periods of the task that demanded increased learning effort. In contrast, model-based control remained unaltered by the influence of hormonal state across the group. Yet, when accounting for individual differences in the genetic proxy of the COMT-Val158Met polymorphism (rs4680), we observed that only the participants homozygote for the methionine allele (n = 12; with putatively higher prefrontal dopamine) experienced a decline in model-based control when facing volatile reward probabilities. This group also showed the increase in suboptimal model-free control, while the carriers of the valine allele remained unaffected by the rise in endogenous estradiol. Taken together, these preliminary findings suggest that endogenous estradiol may affect the balance between model-based and model-free control, and particularly so in women with a high prefrontal baseline dopamine capacity and in situations of increased environmental volatility. |
| first_indexed | 2024-12-19T05:51:35Z |
| format | Article |
| id | doaj.art-8e96950d55bf44ee8d9d51ff82a39475 |
| institution | Directory Open Access Journal |
| issn | 1662-5153 |
| language | English |
| last_indexed | 2024-12-19T05:51:35Z |
| publishDate | 2021-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Behavioral Neuroscience |
| spelling | doaj.art-8e96950d55bf44ee8d9d51ff82a394752022-12-21T20:33:36ZengFrontiers Media S.A.Frontiers in Behavioral Neuroscience1662-51532021-07-011510.3389/fnbeh.2021.658769658769The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based ControlEsther K. Diekhof0Andra Geana1Andra Geana2Frederike Ohm3Bradley B. Doll4Bradley B. Doll5Michael J. Frank6Michael J. Frank7Neuroendocrinology and Human Biology Unit, Department of Biology, Faculty of Mathematics, Informatics and Natural Sciences, Institute of Zoology, Universität Hamburg, Hamburg, GermanyDepartment of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI, United StatesCarney Institute for Brain Science, Brown University, Providence, RI, United StatesNeuroendocrinology and Human Biology Unit, Department of Biology, Faculty of Mathematics, Informatics and Natural Sciences, Institute of Zoology, Universität Hamburg, Hamburg, GermanyNew York University, New York, NY, United StatesColumbia University, New York, NY, United StatesDepartment of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI, United StatesCarney Institute for Brain Science, Brown University, Providence, RI, United StatesThe sex hormone estradiol has recently gained attention in human decision-making research. Animal studies have already shown that estradiol promotes dopaminergic transmission and thus supports reward-seeking behavior and aspects of addiction. In humans, natural variations of estradiol across the menstrual cycle modulate the ability to learn from direct performance feedback (“model-free” learning). However, it remains unclear whether estradiol also influences more complex “model-based” contributions to reinforcement learning. Here, 41 women were tested twice – in the low and high estradiol state of the follicular phase of their menstrual cycle – with a Two-Step decision task designed to separate model-free from model-based learning. The results showed that in the high estradiol state women relied more heavily on model-free learning, and accomplished reduced performance gains, particularly during the more volatile periods of the task that demanded increased learning effort. In contrast, model-based control remained unaltered by the influence of hormonal state across the group. Yet, when accounting for individual differences in the genetic proxy of the COMT-Val158Met polymorphism (rs4680), we observed that only the participants homozygote for the methionine allele (n = 12; with putatively higher prefrontal dopamine) experienced a decline in model-based control when facing volatile reward probabilities. This group also showed the increase in suboptimal model-free control, while the carriers of the valine allele remained unaffected by the rise in endogenous estradiol. Taken together, these preliminary findings suggest that endogenous estradiol may affect the balance between model-based and model-free control, and particularly so in women with a high prefrontal baseline dopamine capacity and in situations of increased environmental volatility.https://www.frontiersin.org/articles/10.3389/fnbeh.2021.658769/fullreinforcement learningestrogenmenstrual cycledopaminereward learningreward volatility |
| spellingShingle | Esther K. Diekhof Andra Geana Andra Geana Frederike Ohm Bradley B. Doll Bradley B. Doll Michael J. Frank Michael J. Frank The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control Frontiers in Behavioral Neuroscience reinforcement learning estrogen menstrual cycle dopamine reward learning reward volatility |
| title | The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control |
| title_full | The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control |
| title_fullStr | The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control |
| title_full_unstemmed | The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control |
| title_short | The Straw That Broke the Camel’s Back: Natural Variations in 17β-Estradiol and COMT-Val158Met Genotype Interact in the Modulation of Model-Free and Model-Based Control |
| title_sort | straw that broke the camel s back natural variations in 17β estradiol and comt val158met genotype interact in the modulation of model free and model based control |
| topic | reinforcement learning estrogen menstrual cycle dopamine reward learning reward volatility |
| url | https://www.frontiersin.org/articles/10.3389/fnbeh.2021.658769/full |
| work_keys_str_mv | AT estherkdiekhof thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT andrageana thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT andrageana thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT frederikeohm thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT bradleybdoll thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT bradleybdoll thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT michaeljfrank thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT michaeljfrank thestrawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT estherkdiekhof strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT andrageana strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT andrageana strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT frederikeohm strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT bradleybdoll strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT bradleybdoll strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT michaeljfrank strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol AT michaeljfrank strawthatbrokethecamelsbacknaturalvariationsin17bestradiolandcomtval158metgenotypeinteractinthemodulationofmodelfreeandmodelbasedcontrol |