Evaluation of information provided to patients by ChatGPT about chronic diseases in Spanish language
Introduction Artificial intelligence has presented exponential growth in medicine. The ChatGPT language model has been highlighted as a possible source of patient information. This study evaluates the reliability and readability of ChatGPT-generated patient information on chronic diseases in Spanish...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2024-01-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076231224603 |
| _version_ | 1827390834816843776 |
|---|---|
| author | María Juliana Soto-Chávez Marlon Mauricio Bustos Daniel G. Fernández-Ávila Oscar Mauricio Muñoz |
| author_facet | María Juliana Soto-Chávez Marlon Mauricio Bustos Daniel G. Fernández-Ávila Oscar Mauricio Muñoz |
| author_sort | María Juliana Soto-Chávez |
| collection | DOAJ |
| description | Introduction Artificial intelligence has presented exponential growth in medicine. The ChatGPT language model has been highlighted as a possible source of patient information. This study evaluates the reliability and readability of ChatGPT-generated patient information on chronic diseases in Spanish. Methods Questions frequently asked by patients on the internet about diabetes mellitus, heart failure, rheumatoid arthritis (RA), chronic kidney disease (CKD), and systemic lupus erythematosus (SLE) were submitted to ChatGPT. Reliability was assessed by rating responses as (1) comprehensive, (2) correct but inadequate, (3) some correct and some incorrect, (4) completely incorrect, and divided between “good” (1 and 2) and “bad” (3 and 4). Readability was evaluated with the adapted Flesch and Szigriszt formulas. Results And 71.67% of the answers were “good,” with none qualified as “completely incorrect.” Better reliability was observed in questions on diabetes and RA versus heart failure (p = 0.02). In readability, responses were “moderately difficult” (54.73, interquartile range (IQR) 51.59–58.58), with better results for CKD (median 56.1, IQR 53.5–59.1) and RA (56.4, IQR 53.7–60.7), than for heart failure responses (median 50.6, IQR 46.3–53.8). Conclusion Our study suggests that the ChatGPT tool can be a reliable source of information in spanish for patients with chronic diseases with different reliability for some of them, however, it needs to improve the readability of its answers to be recommended as a useful tool for patients. |
| first_indexed | 2024-03-08T16:58:35Z |
| format | Article |
| id | doaj.art-293e9ca2d2074c9a9c8bf469e88fd219 |
| institution | Directory Open Access Journal |
| issn | 2055-2076 |
| language | English |
| last_indexed | 2024-03-08T16:58:35Z |
| publishDate | 2024-01-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Digital Health |
| spelling | doaj.art-293e9ca2d2074c9a9c8bf469e88fd2192024-01-04T16:07:25ZengSAGE PublishingDigital Health2055-20762024-01-011010.1177/20552076231224603Evaluation of information provided to patients by ChatGPT about chronic diseases in Spanish languageMaría Juliana Soto-Chávez0Marlon Mauricio Bustos1 Daniel G. Fernández-Ávila2Oscar Mauricio Muñoz3 Department of Internal Medicine, , Bogotá, Colombia Department of Internal Medicine, , Bogotá, Colombia Rheumatology Unit, , Bogotá, Colombia Department of Internal Medicine, , Bogotá, ColombiaIntroduction Artificial intelligence has presented exponential growth in medicine. The ChatGPT language model has been highlighted as a possible source of patient information. This study evaluates the reliability and readability of ChatGPT-generated patient information on chronic diseases in Spanish. Methods Questions frequently asked by patients on the internet about diabetes mellitus, heart failure, rheumatoid arthritis (RA), chronic kidney disease (CKD), and systemic lupus erythematosus (SLE) were submitted to ChatGPT. Reliability was assessed by rating responses as (1) comprehensive, (2) correct but inadequate, (3) some correct and some incorrect, (4) completely incorrect, and divided between “good” (1 and 2) and “bad” (3 and 4). Readability was evaluated with the adapted Flesch and Szigriszt formulas. Results And 71.67% of the answers were “good,” with none qualified as “completely incorrect.” Better reliability was observed in questions on diabetes and RA versus heart failure (p = 0.02). In readability, responses were “moderately difficult” (54.73, interquartile range (IQR) 51.59–58.58), with better results for CKD (median 56.1, IQR 53.5–59.1) and RA (56.4, IQR 53.7–60.7), than for heart failure responses (median 50.6, IQR 46.3–53.8). Conclusion Our study suggests that the ChatGPT tool can be a reliable source of information in spanish for patients with chronic diseases with different reliability for some of them, however, it needs to improve the readability of its answers to be recommended as a useful tool for patients.https://doi.org/10.1177/20552076231224603 |
| spellingShingle | María Juliana Soto-Chávez Marlon Mauricio Bustos Daniel G. Fernández-Ávila Oscar Mauricio Muñoz Evaluation of information provided to patients by ChatGPT about chronic diseases in Spanish language Digital Health |
| title | Evaluation of information provided to patients
by ChatGPT about chronic diseases in
Spanish language |
| title_full | Evaluation of information provided to patients
by ChatGPT about chronic diseases in
Spanish language |
| title_fullStr | Evaluation of information provided to patients
by ChatGPT about chronic diseases in
Spanish language |
| title_full_unstemmed | Evaluation of information provided to patients
by ChatGPT about chronic diseases in
Spanish language |
| title_short | Evaluation of information provided to patients
by ChatGPT about chronic diseases in
Spanish language |
| title_sort | evaluation of information provided to patients by chatgpt about chronic diseases in spanish language |
| url | https://doi.org/10.1177/20552076231224603 |
| work_keys_str_mv | AT mariajulianasotochavez evaluationofinformationprovidedtopatientsbychatgptaboutchronicdiseasesinspanishlanguage AT marlonmauriciobustos evaluationofinformationprovidedtopatientsbychatgptaboutchronicdiseasesinspanishlanguage AT danielgfernandezavila evaluationofinformationprovidedtopatientsbychatgptaboutchronicdiseasesinspanishlanguage AT oscarmauriciomunoz evaluationofinformationprovidedtopatientsbychatgptaboutchronicdiseasesinspanishlanguage |