Can ChatGPT support prospective teachers in physics task development?
The recent advancement of large language models presents numerous opportunities for teaching and learning. Despite widespread public debate regarding the use of large language models, empirical research on their opportunities and risks in education remains limited. In this work, we demonstrate the q...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Physical Society
2023-09-01
|
| Series: | Physical Review Physics Education Research |
| Online Access: | http://doi.org/10.1103/PhysRevPhysEducRes.19.020128 |
| _version_ | 1797688551328972800 |
|---|---|
| author | Stefan Küchemann Steffen Steinert Natalia Revenga Matthias Schweinberger Yavuz Dinc Karina E. Avila Jochen Kuhn |
| author_facet | Stefan Küchemann Steffen Steinert Natalia Revenga Matthias Schweinberger Yavuz Dinc Karina E. Avila Jochen Kuhn |
| author_sort | Stefan Küchemann |
| collection | DOAJ |
| description | The recent advancement of large language models presents numerous opportunities for teaching and learning. Despite widespread public debate regarding the use of large language models, empirical research on their opportunities and risks in education remains limited. In this work, we demonstrate the qualities and shortcomings of using ChatGPT 3.5 for physics task development by prospective teachers. In a randomized controlled trial, 26 prospective physics teacher students were divided into two groups: the first group used ChatGPT 3.5 to develop text-based physics tasks for four different concepts in the field of kinematics for 10th-grade high school students, while the second group used a classical textbook to create tasks for the same concepts and target group. The results indicate no difference in task correctness, but students using the textbook achieved a higher clarity and more frequently embedded their questions in a meaningful context. Both groups adapted the level of task difficulty easily to the target group but struggled strongly with sufficient task specificity, i.e., relevant information to solve the tasks was missing. Students using ChatGPT for problem posing rated high system usability but experienced difficulties with output quality. These results provide insights into the opportunities and pitfalls of using large language models in education. |
| first_indexed | 2024-03-12T01:33:30Z |
| format | Article |
| id | doaj.art-b20c014f7a264981b126e39df05b4e1d |
| institution | Directory Open Access Journal |
| issn | 2469-9896 |
| language | English |
| last_indexed | 2024-03-12T01:33:30Z |
| publishDate | 2023-09-01 |
| publisher | American Physical Society |
| record_format | Article |
| series | Physical Review Physics Education Research |
| spelling | doaj.art-b20c014f7a264981b126e39df05b4e1d2023-09-11T14:27:47ZengAmerican Physical SocietyPhysical Review Physics Education Research2469-98962023-09-0119202012810.1103/PhysRevPhysEducRes.19.020128Can ChatGPT support prospective teachers in physics task development?Stefan KüchemannSteffen SteinertNatalia RevengaMatthias SchweinbergerYavuz DincKarina E. AvilaJochen KuhnThe recent advancement of large language models presents numerous opportunities for teaching and learning. Despite widespread public debate regarding the use of large language models, empirical research on their opportunities and risks in education remains limited. In this work, we demonstrate the qualities and shortcomings of using ChatGPT 3.5 for physics task development by prospective teachers. In a randomized controlled trial, 26 prospective physics teacher students were divided into two groups: the first group used ChatGPT 3.5 to develop text-based physics tasks for four different concepts in the field of kinematics for 10th-grade high school students, while the second group used a classical textbook to create tasks for the same concepts and target group. The results indicate no difference in task correctness, but students using the textbook achieved a higher clarity and more frequently embedded their questions in a meaningful context. Both groups adapted the level of task difficulty easily to the target group but struggled strongly with sufficient task specificity, i.e., relevant information to solve the tasks was missing. Students using ChatGPT for problem posing rated high system usability but experienced difficulties with output quality. These results provide insights into the opportunities and pitfalls of using large language models in education.http://doi.org/10.1103/PhysRevPhysEducRes.19.020128 |
| spellingShingle | Stefan Küchemann Steffen Steinert Natalia Revenga Matthias Schweinberger Yavuz Dinc Karina E. Avila Jochen Kuhn Can ChatGPT support prospective teachers in physics task development? Physical Review Physics Education Research |
| title | Can ChatGPT support prospective teachers in physics task development? |
| title_full | Can ChatGPT support prospective teachers in physics task development? |
| title_fullStr | Can ChatGPT support prospective teachers in physics task development? |
| title_full_unstemmed | Can ChatGPT support prospective teachers in physics task development? |
| title_short | Can ChatGPT support prospective teachers in physics task development? |
| title_sort | can chatgpt support prospective teachers in physics task development |
| url | http://doi.org/10.1103/PhysRevPhysEducRes.19.020128 |
| work_keys_str_mv | AT stefankuchemann canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT steffensteinert canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT nataliarevenga canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT matthiasschweinberger canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT yavuzdinc canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT karinaeavila canchatgptsupportprospectiveteachersinphysicstaskdevelopment AT jochenkuhn canchatgptsupportprospectiveteachersinphysicstaskdevelopment |