Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study
BackgroundDecision support systems (DSSs) for suggesting optimal treatments for individual patients with low back pain (LBP) are currently insufficiently accurate for clinical application. Most of the input provided to train these systems is based on patient-reported outcome...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2024-01-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2024/1/e46857 |
| _version_ | 1797338619441053696 |
|---|---|
| author | Sebastian Fudickar Carsten Bantel Jannik Spieker Heinrich Töpfer Patrick Stegeman Henrica R Schiphorst Preuper Michiel F Reneman André P Wolff Remko Soer |
| author_facet | Sebastian Fudickar Carsten Bantel Jannik Spieker Heinrich Töpfer Patrick Stegeman Henrica R Schiphorst Preuper Michiel F Reneman André P Wolff Remko Soer |
| author_sort | Sebastian Fudickar |
| collection | DOAJ |
| description |
BackgroundDecision support systems (DSSs) for suggesting optimal treatments for individual patients with low back pain (LBP) are currently insufficiently accurate for clinical application. Most of the input provided to train these systems is based on patient-reported outcome measures. However, with the appearance of electronic health records (EHRs), additional qualitative data on reasons for referrals and patients’ goals become available for DSSs. Currently, no decision support tools cover a wide range of biopsychosocial factors, including referral letter information to help clinicians triage patients to the optimal LBP treatment.
ObjectiveThe objective of this study was to investigate the added value of including qualitative data from EHRs and referral letters to the accuracy of a quantitative DSS for patients with LBP.
MethodsA retrospective study was conducted in a clinical cohort of Dutch patients with LBP. Patients filled out a baseline questionnaire about demographics, pain, disability, work status, quality of life, medication, psychosocial functioning, comorbidity, history, and duration of pain. Referral reasons and patient requests for help (patient goals) were extracted via natural language processing (NLP) and enriched in the data set. For decision support, these data were considered independent factors for triage to neurosurgery, anesthesiology, rehabilitation, or minimal intervention. Support vector machine, k-nearest neighbor, and multilayer perceptron models were trained for 2 conditions: with and without consideration of the referral letter content. The models’ accuracies were evaluated via F1-scores, and confusion matrices were used to predict the treatment path (out of 4 paths) with and without additional referral parameters.
ResultsData from 1608 patients were evaluated. The evaluation indicated that 2 referral reasons from the referral letters (for anesthesiology and rehabilitation intervention) increased the F1-score accuracy by up to 19.5% for triaging. The confusion matrices confirmed the results.
ConclusionsThis study indicates that data enriching by adding NLP-based extraction of the content of referral letters increases the model accuracy of DSSs in suggesting optimal treatments for individual patients with LBP. Overall model accuracies were considered low and insufficient for clinical application. |
| first_indexed | 2024-03-08T09:33:55Z |
| format | Article |
| id | doaj.art-791aa26900b847eb898c4fc04c21d535 |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-08T09:33:55Z |
| publishDate | 2024-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-791aa26900b847eb898c4fc04c21d5352024-01-30T15:30:32ZengJMIR PublicationsJournal of Medical Internet Research1438-88712024-01-0126e4685710.2196/46857Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective StudySebastian Fudickarhttps://orcid.org/0000-0002-3553-5131Carsten Bantelhttps://orcid.org/0000-0002-1165-8081Jannik Spiekerhttps://orcid.org/0009-0007-8713-7120Heinrich Töpferhttps://orcid.org/0009-0007-3422-3500Patrick Stegemanhttps://orcid.org/0000-0003-1632-2197Henrica R Schiphorst Preuperhttps://orcid.org/0000-0001-9460-3459Michiel F Renemanhttps://orcid.org/0000-0002-4058-3437André P Wolffhttps://orcid.org/0000-0001-6240-3903Remko Soerhttps://orcid.org/0000-0003-4154-1711 BackgroundDecision support systems (DSSs) for suggesting optimal treatments for individual patients with low back pain (LBP) are currently insufficiently accurate for clinical application. Most of the input provided to train these systems is based on patient-reported outcome measures. However, with the appearance of electronic health records (EHRs), additional qualitative data on reasons for referrals and patients’ goals become available for DSSs. Currently, no decision support tools cover a wide range of biopsychosocial factors, including referral letter information to help clinicians triage patients to the optimal LBP treatment. ObjectiveThe objective of this study was to investigate the added value of including qualitative data from EHRs and referral letters to the accuracy of a quantitative DSS for patients with LBP. MethodsA retrospective study was conducted in a clinical cohort of Dutch patients with LBP. Patients filled out a baseline questionnaire about demographics, pain, disability, work status, quality of life, medication, psychosocial functioning, comorbidity, history, and duration of pain. Referral reasons and patient requests for help (patient goals) were extracted via natural language processing (NLP) and enriched in the data set. For decision support, these data were considered independent factors for triage to neurosurgery, anesthesiology, rehabilitation, or minimal intervention. Support vector machine, k-nearest neighbor, and multilayer perceptron models were trained for 2 conditions: with and without consideration of the referral letter content. The models’ accuracies were evaluated via F1-scores, and confusion matrices were used to predict the treatment path (out of 4 paths) with and without additional referral parameters. ResultsData from 1608 patients were evaluated. The evaluation indicated that 2 referral reasons from the referral letters (for anesthesiology and rehabilitation intervention) increased the F1-score accuracy by up to 19.5% for triaging. The confusion matrices confirmed the results. ConclusionsThis study indicates that data enriching by adding NLP-based extraction of the content of referral letters increases the model accuracy of DSSs in suggesting optimal treatments for individual patients with LBP. Overall model accuracies were considered low and insufficient for clinical application.https://www.jmir.org/2024/1/e46857 |
| spellingShingle | Sebastian Fudickar Carsten Bantel Jannik Spieker Heinrich Töpfer Patrick Stegeman Henrica R Schiphorst Preuper Michiel F Reneman André P Wolff Remko Soer Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study Journal of Medical Internet Research |
| title | Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study |
| title_full | Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study |
| title_fullStr | Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study |
| title_full_unstemmed | Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study |
| title_short | Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study |
| title_sort | natural language processing of referral letters for machine learning based triaging of patients with low back pain to the most appropriate intervention retrospective study |
| url | https://www.jmir.org/2024/1/e46857 |
| work_keys_str_mv | AT sebastianfudickar naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT carstenbantel naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT jannikspieker naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT heinrichtopfer naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT patrickstegeman naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT henricarschiphorstpreuper naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT michielfreneman naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT andrepwolff naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy AT remkosoer naturallanguageprocessingofreferrallettersformachinelearningbasedtriagingofpatientswithlowbackpaintothemostappropriateinterventionretrospectivestudy |