The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study
Abstract Background Neuropsychiatric symptoms (NPS) are prevalent in the early clinical stages of Alzheimer’s disease (AD) according to proxy-based instruments. Little is known about which NPS clinicians report and whether their judgment aligns with proxy-based instruments. We used natural language...
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-05-01
|
| Series: | Alzheimer’s Research & Therapy |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13195-023-01240-7 |
| _version_ | 1797827636390526976 |
|---|---|
| author | Willem S. Eikelboom Ellen H. Singleton Esther van den Berg Casper de Boer Michiel Coesmans Jeannette A. Goudzwaard Everard G. B. Vijverberg Michel Pan Cornalijn Gouw Merel O. Mol Freek Gillissen Jay L. P. Fieldhouse Yolande A. L. Pijnenburg Wiesje M. van der Flier John C. van Swieten Rik Ossenkoppele Jan A. Kors Janne M. Papma |
| author_facet | Willem S. Eikelboom Ellen H. Singleton Esther van den Berg Casper de Boer Michiel Coesmans Jeannette A. Goudzwaard Everard G. B. Vijverberg Michel Pan Cornalijn Gouw Merel O. Mol Freek Gillissen Jay L. P. Fieldhouse Yolande A. L. Pijnenburg Wiesje M. van der Flier John C. van Swieten Rik Ossenkoppele Jan A. Kors Janne M. Papma |
| author_sort | Willem S. Eikelboom |
| collection | DOAJ |
| description | Abstract Background Neuropsychiatric symptoms (NPS) are prevalent in the early clinical stages of Alzheimer’s disease (AD) according to proxy-based instruments. Little is known about which NPS clinicians report and whether their judgment aligns with proxy-based instruments. We used natural language processing (NLP) to classify NPS in electronic health records (EHRs) to estimate the reporting of NPS in symptomatic AD at the memory clinic according to clinicians. Next, we compared NPS as reported in EHRs and NPS reported by caregivers on the Neuropsychiatric Inventory (NPI). Methods Two academic memory clinic cohorts were used: the Amsterdam UMC (n = 3001) and the Erasmus MC (n = 646). Patients included in these cohorts had MCI, AD dementia, or mixed AD/VaD dementia. Ten trained clinicians annotated 13 types of NPS in a randomly selected training set of n = 500 EHRs from the Amsterdam UMC cohort and in a test set of n = 250 EHRs from the Erasmus MC cohort. For each NPS, a generalized linear classifier was trained and internally and externally validated. Prevalence estimates of NPS were adjusted for the imperfect sensitivity and specificity of each classifier. Intra-individual comparison of the NPS classified in EHRs and NPS reported on the NPI were conducted in a subsample (59%). Results Internal validation performance of the classifiers was excellent (AUC range: 0.81–0.91), but external validation performance decreased (AUC range: 0.51–0.93). NPS were prevalent in EHRs from the Amsterdam UMC, especially apathy (adjusted prevalence = 69.4%), anxiety (adjusted prevalence = 53.7%), aberrant motor behavior (adjusted prevalence = 47.5%), irritability (adjusted prevalence = 42.6%), and depression (adjusted prevalence = 38.5%). The ranking of NPS was similar for EHRs from the Erasmus MC, although not all classifiers obtained valid prevalence estimates due to low specificity. In both cohorts, there was minimal agreement between NPS classified in the EHRs and NPS reported on the NPI (all kappa coefficients < 0.28), with substantially more reports of NPS in EHRs than on NPI assessments. Conclusions NLP classifiers performed well in detecting a wide range of NPS in EHRs of patients with symptomatic AD visiting the memory clinic and showed that clinicians frequently reported NPS in these EHRs. Clinicians generally reported more NPS in EHRs than caregivers reported on the NPI. |
| first_indexed | 2024-04-09T12:51:31Z |
| format | Article |
| id | doaj.art-4d57b129f0664ccaaf7252152d89c039 |
| institution | Directory Open Access Journal |
| issn | 1758-9193 |
| language | English |
| last_indexed | 2024-04-09T12:51:31Z |
| publishDate | 2023-05-01 |
| publisher | BMC |
| record_format | Article |
| series | Alzheimer’s Research & Therapy |
| spelling | doaj.art-4d57b129f0664ccaaf7252152d89c0392023-05-14T11:10:41ZengBMCAlzheimer’s Research & Therapy1758-91932023-05-0115111210.1186/s13195-023-01240-7The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing studyWillem S. Eikelboom0Ellen H. Singleton1Esther van den Berg2Casper de Boer3Michiel Coesmans4Jeannette A. Goudzwaard5Everard G. B. Vijverberg6Michel Pan7Cornalijn Gouw8Merel O. Mol9Freek Gillissen10Jay L. P. Fieldhouse11Yolande A. L. Pijnenburg12Wiesje M. van der Flier13John C. van Swieten14Rik Ossenkoppele15Jan A. Kors16Janne M. Papma17Department of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Psychiatry, Erasmus MC University Medical CenterDepartment of Internal Medicine, Section of Geriatrics, Erasmus MC University Medical CenterDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterDepartment of Psychiatry, Erasmus MC University Medical CenterDepartment of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterDepartment of Neurology, Alzheimer Center Amsterdam, Amsterdam University Medical CentersDepartment of Medical Informatics, Erasmus MC University Medical CenterDepartment of Neurology and Alzheimer Center Erasmus MC, Erasmus MC University Medical CenterAbstract Background Neuropsychiatric symptoms (NPS) are prevalent in the early clinical stages of Alzheimer’s disease (AD) according to proxy-based instruments. Little is known about which NPS clinicians report and whether their judgment aligns with proxy-based instruments. We used natural language processing (NLP) to classify NPS in electronic health records (EHRs) to estimate the reporting of NPS in symptomatic AD at the memory clinic according to clinicians. Next, we compared NPS as reported in EHRs and NPS reported by caregivers on the Neuropsychiatric Inventory (NPI). Methods Two academic memory clinic cohorts were used: the Amsterdam UMC (n = 3001) and the Erasmus MC (n = 646). Patients included in these cohorts had MCI, AD dementia, or mixed AD/VaD dementia. Ten trained clinicians annotated 13 types of NPS in a randomly selected training set of n = 500 EHRs from the Amsterdam UMC cohort and in a test set of n = 250 EHRs from the Erasmus MC cohort. For each NPS, a generalized linear classifier was trained and internally and externally validated. Prevalence estimates of NPS were adjusted for the imperfect sensitivity and specificity of each classifier. Intra-individual comparison of the NPS classified in EHRs and NPS reported on the NPI were conducted in a subsample (59%). Results Internal validation performance of the classifiers was excellent (AUC range: 0.81–0.91), but external validation performance decreased (AUC range: 0.51–0.93). NPS were prevalent in EHRs from the Amsterdam UMC, especially apathy (adjusted prevalence = 69.4%), anxiety (adjusted prevalence = 53.7%), aberrant motor behavior (adjusted prevalence = 47.5%), irritability (adjusted prevalence = 42.6%), and depression (adjusted prevalence = 38.5%). The ranking of NPS was similar for EHRs from the Erasmus MC, although not all classifiers obtained valid prevalence estimates due to low specificity. In both cohorts, there was minimal agreement between NPS classified in the EHRs and NPS reported on the NPI (all kappa coefficients < 0.28), with substantially more reports of NPS in EHRs than on NPI assessments. Conclusions NLP classifiers performed well in detecting a wide range of NPS in EHRs of patients with symptomatic AD visiting the memory clinic and showed that clinicians frequently reported NPS in these EHRs. Clinicians generally reported more NPS in EHRs than caregivers reported on the NPI.https://doi.org/10.1186/s13195-023-01240-7Alzheimer’s diseaseApathyAffective symptomsDiagnosisMachine learningNeuropsychiatric symptoms |
| spellingShingle | Willem S. Eikelboom Ellen H. Singleton Esther van den Berg Casper de Boer Michiel Coesmans Jeannette A. Goudzwaard Everard G. B. Vijverberg Michel Pan Cornalijn Gouw Merel O. Mol Freek Gillissen Jay L. P. Fieldhouse Yolande A. L. Pijnenburg Wiesje M. van der Flier John C. van Swieten Rik Ossenkoppele Jan A. Kors Janne M. Papma The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study Alzheimer’s Research & Therapy Alzheimer’s disease Apathy Affective symptoms Diagnosis Machine learning Neuropsychiatric symptoms |
| title | The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study |
| title_full | The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study |
| title_fullStr | The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study |
| title_full_unstemmed | The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study |
| title_short | The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study |
| title_sort | reporting of neuropsychiatric symptoms in electronic health records of individuals with alzheimer s disease a natural language processing study |
| topic | Alzheimer’s disease Apathy Affective symptoms Diagnosis Machine learning Neuropsychiatric symptoms |
| url | https://doi.org/10.1186/s13195-023-01240-7 |
| work_keys_str_mv | AT willemseikelboom thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT ellenhsingleton thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT esthervandenberg thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT casperdeboer thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT michielcoesmans thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jeannetteagoudzwaard thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT everardgbvijverberg thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT michelpan thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT cornalijngouw thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT merelomol thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT freekgillissen thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jaylpfieldhouse thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT yolandealpijnenburg thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT wiesjemvanderflier thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT johncvanswieten thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT rikossenkoppele thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT janakors thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jannempapma thereportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT willemseikelboom reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT ellenhsingleton reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT esthervandenberg reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT casperdeboer reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT michielcoesmans reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jeannetteagoudzwaard reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT everardgbvijverberg reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT michelpan reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT cornalijngouw reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT merelomol reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT freekgillissen reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jaylpfieldhouse reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT yolandealpijnenburg reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT wiesjemvanderflier reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT johncvanswieten reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT rikossenkoppele reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT janakors reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy AT jannempapma reportingofneuropsychiatricsymptomsinelectronichealthrecordsofindividualswithalzheimersdiseaseanaturallanguageprocessingstudy |