Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study
This study aimed to compare the knowledge and interpretation ability of ChatGPT, a language model of artificial general intelligence, with those of medical students in Korea by administering a parasitology examination to both ChatGPT and medical students. The examination consisted of 79 items and wa...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Korea Health Personnel Licensing Examination Institute
2023-01-01
|
| Series: | Journal of Educational Evaluation for Health Professions |
| Subjects: | |
| Online Access: | http://www.jeehp.org/upload/jeehp-20-01.pdf |
| _version_ | 1797787297469431808 |
|---|---|
| author | Sun Huh |
| author_facet | Sun Huh |
| author_sort | Sun Huh |
| collection | DOAJ |
| description | This study aimed to compare the knowledge and interpretation ability of ChatGPT, a language model of artificial general intelligence, with those of medical students in Korea by administering a parasitology examination to both ChatGPT and medical students. The examination consisted of 79 items and was administered to ChatGPT on January 1, 2023. The examination results were analyzed in terms of ChatGPT’s overall performance score, its correct answer rate by the items’ knowledge level, and the acceptability of its explanations of the items. ChatGPT’s performance was lower than that of the medical students, and ChatGPT’s correct answer rate was not related to the items’ knowledge level. However, there was a relationship between acceptable explanations and correct answers. In conclusion, ChatGPT’s knowledge and interpretation ability for this parasitology examination were not yet comparable to those of medical students in Korea. |
| first_indexed | 2024-03-13T01:20:54Z |
| format | Article |
| id | doaj.art-f4bb1a1df5e741b58ad1f01cca866493 |
| institution | Directory Open Access Journal |
| issn | 1975-5937 |
| language | English |
| last_indexed | 2024-03-13T01:20:54Z |
| publishDate | 2023-01-01 |
| publisher | Korea Health Personnel Licensing Examination Institute |
| record_format | Article |
| series | Journal of Educational Evaluation for Health Professions |
| spelling | doaj.art-f4bb1a1df5e741b58ad1f01cca8664932023-07-05T04:47:44ZengKorea Health Personnel Licensing Examination InstituteJournal of Educational Evaluation for Health Professions1975-59372023-01-012010.3352/jeehp.2023.20.1465Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive studySun HuhThis study aimed to compare the knowledge and interpretation ability of ChatGPT, a language model of artificial general intelligence, with those of medical students in Korea by administering a parasitology examination to both ChatGPT and medical students. The examination consisted of 79 items and was administered to ChatGPT on January 1, 2023. The examination results were analyzed in terms of ChatGPT’s overall performance score, its correct answer rate by the items’ knowledge level, and the acceptability of its explanations of the items. ChatGPT’s performance was lower than that of the medical students, and ChatGPT’s correct answer rate was not related to the items’ knowledge level. However, there was a relationship between acceptable explanations and correct answers. In conclusion, ChatGPT’s knowledge and interpretation ability for this parasitology examination were not yet comparable to those of medical students in Korea.http://www.jeehp.org/upload/jeehp-20-01.pdfartificial intelligenceeducational measurementknowledgemedical studentsrepublic of korea |
| spellingShingle | Sun Huh Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study Journal of Educational Evaluation for Health Professions artificial intelligence educational measurement knowledge medical students republic of korea |
| title | Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study |
| title_full | Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study |
| title_fullStr | Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study |
| title_full_unstemmed | Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study |
| title_short | Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study |
| title_sort | are chatgpt s knowledge and interpretation ability comparable to those of medical students in korea for taking a parasitology examination a descriptive study |
| topic | artificial intelligence educational measurement knowledge medical students republic of korea |
| url | http://www.jeehp.org/upload/jeehp-20-01.pdf |
| work_keys_str_mv | AT sunhuh arechatgptsknowledgeandinterpretationabilitycomparabletothoseofmedicalstudentsinkoreafortakingaparasitologyexaminationadescriptivestudy |