Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis
Artificial Intelligence (AI) is rapidly transforming various sectors of industry, including the medical community, due to its ability to simulate human intelligence processes. This study evaluates the capacity of an AI tool, ChatGPT 3.5, to summarize scientific papers in the foot and ankle surgery l...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-01-01
|
| Series: | Foot & Ankle Surgery: Techniques, Reports & Cases |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667396724000247 |
| _version_ | 1827279067503656960 |
|---|---|
| author | Steven R. Cooperman, DPM, MBA, AACFAS Roberto A. Brandão, DPM, FACFAS |
| author_facet | Steven R. Cooperman, DPM, MBA, AACFAS Roberto A. Brandão, DPM, FACFAS |
| author_sort | Steven R. Cooperman, DPM, MBA, AACFAS |
| collection | DOAJ |
| description | Artificial Intelligence (AI) is rapidly transforming various sectors of industry, including the medical community, due to its ability to simulate human intelligence processes. This study evaluates the capacity of an AI tool, ChatGPT 3.5, to summarize scientific papers in the foot and ankle surgery literature, comparing its performance to summaries written by podiatric surgery residents. Quantitative and Qualitative analyses were performed, including BLEU and ROUGE metrics, Flesch Reading Ease Score (FRES), Flesch-Kincaide Grade Level (FKGL) readability statistics, qualitative analysis by independent reviewers, and independent accuracy assessments. Results indicate that AI-generated summaries closely resemble those produced by podiatric surgery residents in terms of content, readability, and accuracy. Although differing slightly from their human-generated counterparts with a higher level of writing, AI-generated summaries offer consistency and conciseness. Overall, this study demonstrates the potential of AI tools to streamline research processes while emphasizing the importance of judicious use and oversight to maintain scholarly integrity. |
| first_indexed | 2024-04-24T08:11:54Z |
| format | Article |
| id | doaj.art-899fb20ee228489fadb5602b438fc02a |
| institution | Directory Open Access Journal |
| issn | 2667-3967 |
| language | English |
| last_indexed | 2024-04-24T08:11:54Z |
| publishDate | 2024-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Foot & Ankle Surgery: Techniques, Reports & Cases |
| spelling | doaj.art-899fb20ee228489fadb5602b438fc02a2024-04-17T04:50:15ZengElsevierFoot & Ankle Surgery: Techniques, Reports & Cases2667-39672024-01-0142100384Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysisSteven R. Cooperman, DPM, MBA, AACFAS0Roberto A. Brandão, DPM, FACFAS1Fellow; Orthopedic Foot and Ankle Center Advanced Foot and Ankle Reconstruction Fellowship, 350 W. Wilson Bridge Rd, Ste. 200, Worthington, OH 43085, USA; Corresponding author at: 350 W. Wilson Bridge Rd, Ste. 200, Worthington, OH 43085, USA.Faculty; Orthopedic Foot and Ankle Center Advanced Foot and Ankle Reconstruction Fellowship, 350 W. Wilson Bridge Rd, Ste. 200, Worthington, OH 43085, USAArtificial Intelligence (AI) is rapidly transforming various sectors of industry, including the medical community, due to its ability to simulate human intelligence processes. This study evaluates the capacity of an AI tool, ChatGPT 3.5, to summarize scientific papers in the foot and ankle surgery literature, comparing its performance to summaries written by podiatric surgery residents. Quantitative and Qualitative analyses were performed, including BLEU and ROUGE metrics, Flesch Reading Ease Score (FRES), Flesch-Kincaide Grade Level (FKGL) readability statistics, qualitative analysis by independent reviewers, and independent accuracy assessments. Results indicate that AI-generated summaries closely resemble those produced by podiatric surgery residents in terms of content, readability, and accuracy. Although differing slightly from their human-generated counterparts with a higher level of writing, AI-generated summaries offer consistency and conciseness. Overall, this study demonstrates the potential of AI tools to streamline research processes while emphasizing the importance of judicious use and oversight to maintain scholarly integrity.http://www.sciencedirect.com/science/article/pii/S2667396724000247III, comparative study |
| spellingShingle | Steven R. Cooperman, DPM, MBA, AACFAS Roberto A. Brandão, DPM, FACFAS Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis Foot & Ankle Surgery: Techniques, Reports & Cases III, comparative study |
| title | Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis |
| title_full | Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis |
| title_fullStr | Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis |
| title_full_unstemmed | Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis |
| title_short | Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis |
| title_sort | investigating the proficiency of an ai tool in summarizing foot and ankle literature a quantitative qualitative and accuracy analysis |
| topic | III, comparative study |
| url | http://www.sciencedirect.com/science/article/pii/S2667396724000247 |
| work_keys_str_mv | AT stevenrcoopermandpmmbaaacfas investigatingtheproficiencyofanaitoolinsummarizingfootandankleliteratureaquantitativequalitativeandaccuracyanalysis AT robertoabrandaodpmfacfas investigatingtheproficiencyofanaitoolinsummarizingfootandankleliteratureaquantitativequalitativeandaccuracyanalysis |