Partnering with AI for instrument development: possibilities and pitfalls
Recent advances in generative artificial intelligence (AI), specifically large language models (LLMs), provide new possibilities for researchers to partner with AI when developing and refining psychological instruments. In this paper we demonstrate how LLMs, such as OpenAI's ChatGPT 4 model, mi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025
|
| Subjects: | |
| Online Access: | https://repository.londonmet.ac.uk/9679/1/Beghetto-et-al-2024-Partnering-with-AI.pdf |
| _version_ | 1824446486451060736 |
|---|---|
| author | Beghetto, Ronald A. Ross, Wendy Karwowski, Maciej Glăveanu, Vlad |
| author_facet | Beghetto, Ronald A. Ross, Wendy Karwowski, Maciej Glăveanu, Vlad |
| author_sort | Beghetto, Ronald A. |
| collection | LMU |
| description | Recent advances in generative artificial intelligence (AI), specifically large language models (LLMs), provide new possibilities for researchers to partner with AI when developing and refining psychological instruments. In this paper we demonstrate how LLMs, such as OpenAI's ChatGPT 4 model, might be used to support the development of new psychometric scales. Partnering with AI for the purpose of developing and refining instruments, however, comes with its share of potential pitfalls. We thereby discuss throughout the paper that instrument development and refinement start and end with human judgment and expertise. We open with two use-cases that describe how we used LLMs in the development and refinement of two new psychological instruments. Next, we discuss possibilities for where and how researchers can use LLMs in the process of instrument development more broadly, including considerations for maximizing the benefits of LLMs and addressing the potential hazards when working with LLMs. Finally, we close by offering initial suggestions for psychology researchers interested in partnering with LLMs in this capacity. |
| first_indexed | 2025-02-19T01:15:55Z |
| format | Article |
| id | oai:repository.londonmet.ac.uk:9679 |
| institution | London Metropolitan University |
| language | English |
| last_indexed | 2025-02-19T01:15:55Z |
| publishDate | 2025 |
| publisher | Elsevier |
| record_format | eprints |
| spelling | oai:repository.londonmet.ac.uk:96792024-09-23T08:17:45Z https://repository.londonmet.ac.uk/9679/ Partnering with AI for instrument development: possibilities and pitfalls Beghetto, Ronald A. Ross, Wendy Karwowski, Maciej Glăveanu, Vlad 000 Computer science, information & general works 150 Psychology Recent advances in generative artificial intelligence (AI), specifically large language models (LLMs), provide new possibilities for researchers to partner with AI when developing and refining psychological instruments. In this paper we demonstrate how LLMs, such as OpenAI's ChatGPT 4 model, might be used to support the development of new psychometric scales. Partnering with AI for the purpose of developing and refining instruments, however, comes with its share of potential pitfalls. We thereby discuss throughout the paper that instrument development and refinement start and end with human judgment and expertise. We open with two use-cases that describe how we used LLMs in the development and refinement of two new psychological instruments. Next, we discuss possibilities for where and how researchers can use LLMs in the process of instrument development more broadly, including considerations for maximizing the benefits of LLMs and addressing the potential hazards when working with LLMs. Finally, we close by offering initial suggestions for psychology researchers interested in partnering with LLMs in this capacity. Elsevier 2025-01 Article PeerReviewed text en cc_by_nc_nd_4 https://repository.londonmet.ac.uk/9679/1/Beghetto-et-al-2024-Partnering-with-AI.pdf Beghetto, Ronald A., Ross, Wendy, Karwowski, Maciej and Glăveanu, Vlad (2025) Partnering with AI for instrument development: possibilities and pitfalls. New Ideas in Psychology, 76 (101121). pp. 1-10. ISSN 0732-118X https://doi.org/10.1016/j.newideapsych.2024.101121 10.1016/j.newideapsych.2024.101121 10.1016/j.newideapsych.2024.101121 |
| spellingShingle | 000 Computer science, information & general works 150 Psychology Beghetto, Ronald A. Ross, Wendy Karwowski, Maciej Glăveanu, Vlad Partnering with AI for instrument development: possibilities and pitfalls |
| title | Partnering with AI for instrument development: possibilities and pitfalls |
| title_full | Partnering with AI for instrument development: possibilities and pitfalls |
| title_fullStr | Partnering with AI for instrument development: possibilities and pitfalls |
| title_full_unstemmed | Partnering with AI for instrument development: possibilities and pitfalls |
| title_short | Partnering with AI for instrument development: possibilities and pitfalls |
| title_sort | partnering with ai for instrument development possibilities and pitfalls |
| topic | 000 Computer science, information & general works 150 Psychology |
| url | https://repository.londonmet.ac.uk/9679/1/Beghetto-et-al-2024-Partnering-with-AI.pdf |
| work_keys_str_mv | AT beghettoronalda partneringwithaiforinstrumentdevelopmentpossibilitiesandpitfalls AT rosswendy partneringwithaiforinstrumentdevelopmentpossibilitiesandpitfalls AT karwowskimaciej partneringwithaiforinstrumentdevelopmentpossibilitiesandpitfalls AT glaveanuvlad partneringwithaiforinstrumentdevelopmentpossibilitiesandpitfalls |