Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon
Introduction/Purpose: ChatGPT is an Artificial intelligence (AI) algorithm based on a user-friendly interface that does not requires advanced programming skills. Since its release to the public, it has made AI more accessible. As a result, the Patient refers to chat GTP for medical inquiries. This s...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2024-04-01
|
| Series: | Foot & Ankle Orthopaedics |
| Online Access: | https://doi.org/10.1177/2473011424S00045 |
| _version_ | 1797212982246113280 |
|---|---|
| author | Ben Efrima MD Agustin Barbero MD Cristian Indino MD Camilla Maccario MD Federico Giuseppe Usuelli MD |
| author_facet | Ben Efrima MD Agustin Barbero MD Cristian Indino MD Camilla Maccario MD Federico Giuseppe Usuelli MD |
| author_sort | Ben Efrima MD |
| collection | DOAJ |
| description | Introduction/Purpose: ChatGPT is an Artificial intelligence (AI) algorithm based on a user-friendly interface that does not requires advanced programming skills. Since its release to the public, it has made AI more accessible. As a result, the Patient refers to chat GTP for medical inquiries. This study compares the postoperative (PO) information provided by chatGPT to those of a high- volume foot and ankle orthopedics surgeon. Methods: The study includes 251 patients treated for end-stage osteoarthritis with total ankle arthroplasty. Postoperative emails containing inquiries about their PO status were uploaded to chatGTP. We then evaluated the agreement in simple (SA) and complex (CA) answers. The SA abbreviated the answer provided into "yes" or "no" Its agreement analysis was made using Cohens Kappa. In contrast, CA contained detailed information, and answers were classified into complete agreement, partial agreement, or complete disagreement. Additionally, in partial agreement answers, we calculated the percentage of agreement. Finally, we measured the cases where the surgeon added additional information unrelated to the inquiry. Results: There was only a slight agreement in the SA category (K=0.08). In the CA category, we found 27 percent of complete agreement; in 52 percent, we found complete disagreement; in 20 percent, we found only partial agreement. In 50 % of the cases, the surgeon added information unrelated to the question. Conclusion: We found limited agreement between chat GPT and the primary surgeon regarding postoperative information. Indicating that chat, GPT is currently an unreliable source for postoperative management. Table 2 Distribution of agreement percentage between a High-volume foot and ankle surgeon and ChatGPT response to postoperative inquiries from patients with total ankle arthroplasty in those with partial agreement |
| first_indexed | 2024-04-24T10:51:02Z |
| format | Article |
| id | doaj.art-6c14a99bdab5411ba550fb452fdcff24 |
| institution | Directory Open Access Journal |
| issn | 2473-0114 |
| language | English |
| last_indexed | 2024-04-24T10:51:02Z |
| publishDate | 2024-04-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Foot & Ankle Orthopaedics |
| spelling | doaj.art-6c14a99bdab5411ba550fb452fdcff242024-04-12T13:04:06ZengSAGE PublishingFoot & Ankle Orthopaedics2473-01142024-04-01910.1177/2473011424S00045Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle SurgeonBen Efrima MDAgustin Barbero MDCristian Indino MDCamilla Maccario MDFederico Giuseppe Usuelli MDIntroduction/Purpose: ChatGPT is an Artificial intelligence (AI) algorithm based on a user-friendly interface that does not requires advanced programming skills. Since its release to the public, it has made AI more accessible. As a result, the Patient refers to chat GTP for medical inquiries. This study compares the postoperative (PO) information provided by chatGPT to those of a high- volume foot and ankle orthopedics surgeon. Methods: The study includes 251 patients treated for end-stage osteoarthritis with total ankle arthroplasty. Postoperative emails containing inquiries about their PO status were uploaded to chatGTP. We then evaluated the agreement in simple (SA) and complex (CA) answers. The SA abbreviated the answer provided into "yes" or "no" Its agreement analysis was made using Cohens Kappa. In contrast, CA contained detailed information, and answers were classified into complete agreement, partial agreement, or complete disagreement. Additionally, in partial agreement answers, we calculated the percentage of agreement. Finally, we measured the cases where the surgeon added additional information unrelated to the inquiry. Results: There was only a slight agreement in the SA category (K=0.08). In the CA category, we found 27 percent of complete agreement; in 52 percent, we found complete disagreement; in 20 percent, we found only partial agreement. In 50 % of the cases, the surgeon added information unrelated to the question. Conclusion: We found limited agreement between chat GPT and the primary surgeon regarding postoperative information. Indicating that chat, GPT is currently an unreliable source for postoperative management. Table 2 Distribution of agreement percentage between a High-volume foot and ankle surgeon and ChatGPT response to postoperative inquiries from patients with total ankle arthroplasty in those with partial agreementhttps://doi.org/10.1177/2473011424S00045 |
| spellingShingle | Ben Efrima MD Agustin Barbero MD Cristian Indino MD Camilla Maccario MD Federico Giuseppe Usuelli MD Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon Foot & Ankle Orthopaedics |
| title | Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon |
| title_full | Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon |
| title_fullStr | Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon |
| title_full_unstemmed | Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon |
| title_short | Agreement Analysis on Postoperative Care Between ChatGPT and a High-Volume Foot and Ankle Surgeon |
| title_sort | agreement analysis on postoperative care between chatgpt and a high volume foot and ankle surgeon |
| url | https://doi.org/10.1177/2473011424S00045 |
| work_keys_str_mv | AT benefrimamd agreementanalysisonpostoperativecarebetweenchatgptandahighvolumefootandanklesurgeon AT agustinbarberomd agreementanalysisonpostoperativecarebetweenchatgptandahighvolumefootandanklesurgeon AT cristianindinomd agreementanalysisonpostoperativecarebetweenchatgptandahighvolumefootandanklesurgeon AT camillamaccariomd agreementanalysisonpostoperativecarebetweenchatgptandahighvolumefootandanklesurgeon AT federicogiuseppeusuellimd agreementanalysisonpostoperativecarebetweenchatgptandahighvolumefootandanklesurgeon |