Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints
Early identification of acute gout is crucial, enabling healthcare professionals to implement targeted interventions for rapid pain relief and preventing disease progression, ensuring improved long-term joint function. In this study, we comprehensively explored the potential early detection of gout...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Forecasting |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2571-9394/6/1/13 |
| _version_ | 1797241039977709568 |
|---|---|
| author | Lucas Lopes Oliveira Xiaorui Jiang Aryalakshmi Nellippillipathil Babu Poonam Karajagi Alireza Daneshkhah |
| author_facet | Lucas Lopes Oliveira Xiaorui Jiang Aryalakshmi Nellippillipathil Babu Poonam Karajagi Alireza Daneshkhah |
| author_sort | Lucas Lopes Oliveira |
| collection | DOAJ |
| description | Early identification of acute gout is crucial, enabling healthcare professionals to implement targeted interventions for rapid pain relief and preventing disease progression, ensuring improved long-term joint function. In this study, we comprehensively explored the potential early detection of gout flares (GFs) based on nurses’ chief complaint notes in the Emergency Department (ED). Addressing the challenge of identifying GFs prospectively during an ED visit, where documentation is typically minimal, our research focused on employing alternative Natural Language Processing (NLP) techniques to enhance detection accuracy. We investigated GF detection algorithms using both sparse representations by traditional NLP methods and dense encodings by medical domain-specific Large Language Models (LLMs), distinguishing between generative and discriminative models. Three methods were used to alleviate the issue of severe data imbalances, including oversampling, class weights, and focal loss. Extensive empirical studies were performed on the Gout Emergency Department Chief Complaint Corpora. Sparse text representations like tf-idf proved to produce strong performances, achieving F1 scores higher than 0.75. The best deep learning models were RoBERTa-large-PM-M3-Voc and BioGPT, which had the best F1 scores for each dataset, with a 0.8 on the 2019 dataset and a 0.85 F1 score on the 2020 dataset, respectively. We concluded that although discriminative LLMs performed better for this classification task when compared to generative LLMs, a combination of using generative models as feature extractors and employing a support vector machine for classification yielded promising results comparable to those obtained with discriminative models. |
| first_indexed | 2024-04-24T18:17:00Z |
| format | Article |
| id | doaj.art-7932b4182dee462897bc34d8bb0f9901 |
| institution | Directory Open Access Journal |
| issn | 2571-9394 |
| language | English |
| last_indexed | 2024-04-24T18:17:00Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Forecasting |
| spelling | doaj.art-7932b4182dee462897bc34d8bb0f99012024-03-27T13:41:23ZengMDPI AGForecasting2571-93942024-03-016122423810.3390/forecast6010013Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief ComplaintsLucas Lopes Oliveira0Xiaorui Jiang1Aryalakshmi Nellippillipathil Babu2Poonam Karajagi3Alireza Daneshkhah4School of Computing, Mathematics and Data Sciences, Coventry University, Coventry CV1 5FB, UKCentre for Computational Sciences and Mathematical Modelling, Coventry University, Coventry CV1 2TT, UKSchool of Computing, Mathematics and Data Sciences, Coventry University, Coventry CV1 5FB, UKSchool of Computing, Mathematics and Data Sciences, Coventry University, Coventry CV1 5FB, UKSchool of Computing, Mathematics and Data Sciences, Coventry University, Coventry CV1 5FB, UKEarly identification of acute gout is crucial, enabling healthcare professionals to implement targeted interventions for rapid pain relief and preventing disease progression, ensuring improved long-term joint function. In this study, we comprehensively explored the potential early detection of gout flares (GFs) based on nurses’ chief complaint notes in the Emergency Department (ED). Addressing the challenge of identifying GFs prospectively during an ED visit, where documentation is typically minimal, our research focused on employing alternative Natural Language Processing (NLP) techniques to enhance detection accuracy. We investigated GF detection algorithms using both sparse representations by traditional NLP methods and dense encodings by medical domain-specific Large Language Models (LLMs), distinguishing between generative and discriminative models. Three methods were used to alleviate the issue of severe data imbalances, including oversampling, class weights, and focal loss. Extensive empirical studies were performed on the Gout Emergency Department Chief Complaint Corpora. Sparse text representations like tf-idf proved to produce strong performances, achieving F1 scores higher than 0.75. The best deep learning models were RoBERTa-large-PM-M3-Voc and BioGPT, which had the best F1 scores for each dataset, with a 0.8 on the 2019 dataset and a 0.85 F1 score on the 2020 dataset, respectively. We concluded that although discriminative LLMs performed better for this classification task when compared to generative LLMs, a combination of using generative models as feature extractors and employing a support vector machine for classification yielded promising results comparable to those obtained with discriminative models.https://www.mdpi.com/2571-9394/6/1/13gout flarechief complaintnatural language processingdeep learninglarge language models |
| spellingShingle | Lucas Lopes Oliveira Xiaorui Jiang Aryalakshmi Nellippillipathil Babu Poonam Karajagi Alireza Daneshkhah Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints Forecasting gout flare chief complaint natural language processing deep learning large language models |
| title | Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints |
| title_full | Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints |
| title_fullStr | Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints |
| title_full_unstemmed | Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints |
| title_short | Effective Natural Language Processing Algorithms for Early Alerts of Gout Flares from Chief Complaints |
| title_sort | effective natural language processing algorithms for early alerts of gout flares from chief complaints |
| topic | gout flare chief complaint natural language processing deep learning large language models |
| url | https://www.mdpi.com/2571-9394/6/1/13 |
| work_keys_str_mv | AT lucaslopesoliveira effectivenaturallanguageprocessingalgorithmsforearlyalertsofgoutflaresfromchiefcomplaints AT xiaoruijiang effectivenaturallanguageprocessingalgorithmsforearlyalertsofgoutflaresfromchiefcomplaints AT aryalakshminellippillipathilbabu effectivenaturallanguageprocessingalgorithmsforearlyalertsofgoutflaresfromchiefcomplaints AT poonamkarajagi effectivenaturallanguageprocessingalgorithmsforearlyalertsofgoutflaresfromchiefcomplaints AT alirezadaneshkhah effectivenaturallanguageprocessingalgorithmsforearlyalertsofgoutflaresfromchiefcomplaints |