Examining LLMs in Economic Settings
Humans are not homo economicus (i.e., rational economic beings). We exhibit systematic behavioral biases such as loss aversion, anchoring, framing, etc., which lead us to make suboptimal economic decisions. Insofar as such biases may embedded in text data on which large language models (LLMs) are tr...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156339 |
| _version_ | 1826199695667494912 |
|---|---|
| author | Ross, Jillian A. |
| author2 | Lo, Andrew W. |
| author_facet | Lo, Andrew W. Ross, Jillian A. |
| author_sort | Ross, Jillian A. |
| collection | MIT |
| description | Humans are not homo economicus (i.e., rational economic beings). We exhibit systematic behavioral biases such as loss aversion, anchoring, framing, etc., which lead us to make suboptimal economic decisions. Insofar as such biases may embedded in text data on which large language models (LLMs) are trained, to what extent are LLMs prone to the same behavioral biases? Understanding these biases in LLMs is crucial for deploying LLMs to support human decision-making. To enable the responsible deployment of LLMs, I propose economic alignment. Economic alignment is a specific form of AI alignment that provides a critical perspective to interrogate what human preferences we would like to incorporate into LLM decisions. To illustrate the power of economic alignment, I systematically study the economic decision-making behaviors of LLMs through utility theory, a paradigm at the core of modern economic theory. I apply experimental designs from human studies to LLMs and find that they are neither entirely human-like nor entirely economicus-like. Specifically, I find that LLMs generally exhibit stronger inequity aversion, stronger loss aversion, weaker risk aversion, and stronger time discounting compared to human subjects. I further find that most LLMs struggle to maintain consistent economic behavior across settings. Finally, I present a case study that examines how we can intervene through prompting to better align LLMs with economic goals. |
| first_indexed | 2024-09-23T11:24:07Z |
| format | Thesis |
| id | mit-1721.1/156339 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T11:24:07Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1563392024-08-22T03:40:25Z Examining LLMs in Economic Settings Ross, Jillian A. Lo, Andrew W. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Humans are not homo economicus (i.e., rational economic beings). We exhibit systematic behavioral biases such as loss aversion, anchoring, framing, etc., which lead us to make suboptimal economic decisions. Insofar as such biases may embedded in text data on which large language models (LLMs) are trained, to what extent are LLMs prone to the same behavioral biases? Understanding these biases in LLMs is crucial for deploying LLMs to support human decision-making. To enable the responsible deployment of LLMs, I propose economic alignment. Economic alignment is a specific form of AI alignment that provides a critical perspective to interrogate what human preferences we would like to incorporate into LLM decisions. To illustrate the power of economic alignment, I systematically study the economic decision-making behaviors of LLMs through utility theory, a paradigm at the core of modern economic theory. I apply experimental designs from human studies to LLMs and find that they are neither entirely human-like nor entirely economicus-like. Specifically, I find that LLMs generally exhibit stronger inequity aversion, stronger loss aversion, weaker risk aversion, and stronger time discounting compared to human subjects. I further find that most LLMs struggle to maintain consistent economic behavior across settings. Finally, I present a case study that examines how we can intervene through prompting to better align LLMs with economic goals. S.M. 2024-08-21T18:57:56Z 2024-08-21T18:57:56Z 2024-05 2024-07-10T12:59:53.396Z Thesis https://hdl.handle.net/1721.1/156339 Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Ross, Jillian A. Examining LLMs in Economic Settings |
| title | Examining LLMs in Economic Settings |
| title_full | Examining LLMs in Economic Settings |
| title_fullStr | Examining LLMs in Economic Settings |
| title_full_unstemmed | Examining LLMs in Economic Settings |
| title_short | Examining LLMs in Economic Settings |
| title_sort | examining llms in economic settings |
| url | https://hdl.handle.net/1721.1/156339 |
| work_keys_str_mv | AT rossjilliana examiningllmsineconomicsettings |