Working memory capacity of ChatGPT: an empirical study
Working memory is a critical aspect of both human intelligence and artificial intelligence, serving as a workspace for the temporary storage and manipulation of information. In this paper, we systematically assess the working memory capacity of ChatGPT, a large language model developed by OpenAI, by...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Association for the Advancement of Artificial Intelligence
2024
|
| _version_ | 1797113332869627904 |
|---|---|
| author | Gong, D Wan, X Wang, D |
| author_facet | Gong, D Wan, X Wang, D |
| author_sort | Gong, D |
| collection | OXFORD |
| description | Working memory is a critical aspect of both human intelligence and artificial intelligence, serving as a workspace for the temporary storage and manipulation of information. In this paper, we systematically assess the working memory capacity of ChatGPT, a large language model developed by OpenAI, by examining its performance in verbal and spatial n-back tasks under various conditions. Our experiments reveal that ChatGPT has a working memory capacity limit strikingly similar to that of humans. Furthermore, we investigate the impact of different instruction strategies on ChatGPT's performance and observe that the fundamental patterns of a capacity limit persist. From our empirical findings, we propose that n-back tasks may serve as tools for benchmarking the working memory capacity of large language models and hold potential for informing future efforts aimed at enhancing AI working memory. |
| first_indexed | 2024-03-07T08:14:26Z |
| format | Conference item |
| id | oxford-uuid:a7f9a182-b932-417c-804d-320f56956b88 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-04-23T08:27:09Z |
| publishDate | 2024 |
| publisher | Association for the Advancement of Artificial Intelligence |
| record_format | dspace |
| spelling | oxford-uuid:a7f9a182-b932-417c-804d-320f56956b882024-04-19T10:34:39ZWorking memory capacity of ChatGPT: an empirical studyConference itemhttp://purl.org/coar/resource_type/c_5794uuid:a7f9a182-b932-417c-804d-320f56956b88EnglishSymplectic ElementsAssociation for the Advancement of Artificial Intelligence2024Gong, DWan, XWang, DWorking memory is a critical aspect of both human intelligence and artificial intelligence, serving as a workspace for the temporary storage and manipulation of information. In this paper, we systematically assess the working memory capacity of ChatGPT, a large language model developed by OpenAI, by examining its performance in verbal and spatial n-back tasks under various conditions. Our experiments reveal that ChatGPT has a working memory capacity limit strikingly similar to that of humans. Furthermore, we investigate the impact of different instruction strategies on ChatGPT's performance and observe that the fundamental patterns of a capacity limit persist. From our empirical findings, we propose that n-back tasks may serve as tools for benchmarking the working memory capacity of large language models and hold potential for informing future efforts aimed at enhancing AI working memory. |
| spellingShingle | Gong, D Wan, X Wang, D Working memory capacity of ChatGPT: an empirical study |
| title | Working memory capacity of ChatGPT: an empirical study |
| title_full | Working memory capacity of ChatGPT: an empirical study |
| title_fullStr | Working memory capacity of ChatGPT: an empirical study |
| title_full_unstemmed | Working memory capacity of ChatGPT: an empirical study |
| title_short | Working memory capacity of ChatGPT: an empirical study |
| title_sort | working memory capacity of chatgpt an empirical study |
| work_keys_str_mv | AT gongd workingmemorycapacityofchatgptanempiricalstudy AT wanx workingmemorycapacityofchatgptanempiricalstudy AT wangd workingmemorycapacityofchatgptanempiricalstudy |