The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records
BackgroundAffective characteristics are associated with depression severity, course, and prognosis. Patients’ affect captured by clinicians during sessions may provide a rich source of information that more naturally aligns with the depression course and patient-desired depre...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-05-01
|
| Series: | JMIR Formative Research |
| Online Access: | https://formative.jmir.org/2022/5/e34436 |
| _version_ | 1827858280543682560 |
|---|---|
| author | Vanessa Panaite Andrew R Devendorf Dezon Finch Lina Bouayad Stephen L Luther Susan K Schultz |
| author_facet | Vanessa Panaite Andrew R Devendorf Dezon Finch Lina Bouayad Stephen L Luther Susan K Schultz |
| author_sort | Vanessa Panaite |
| collection | DOAJ |
| description |
BackgroundAffective characteristics are associated with depression severity, course, and prognosis. Patients’ affect captured by clinicians during sessions may provide a rich source of information that more naturally aligns with the depression course and patient-desired depression outcomes.
ObjectiveIn this paper, we propose an information extraction vocabulary used to pilot the feasibility and reliability of identifying clinician-recorded patient affective states in clinical notes from electronic health records.
MethodsAffect and mood were annotated in 147 clinical notes of 109 patients by 2 independent coders across 3 pilots. Intercoder discrepancies were settled by a third coder. This reference annotation set was used to test a proof-of-concept natural language processing (NLP) system using a named entity recognition approach.
ResultsConcepts were frequently addressed in templated format and free text in clinical notes. Annotated data demonstrated that affective characteristics were identified in 87.8% (129/147) of the notes, while mood was identified in 97.3% (143/147) of the notes. The intercoder reliability was consistently good across the pilots (interannotator agreement [IAA] >70%). The final NLP system showed good reliability with the final reference annotation set (mood IAA=85.8%; affect IAA=80.9%).
ConclusionsAffect and mood can be reliably identified in clinician reports and are good targets for NLP. We discuss several next steps to expand on this proof of concept and the value of this research for depression clinical research. |
| first_indexed | 2024-03-12T12:53:36Z |
| format | Article |
| id | doaj.art-7d6ea91491484b1799ee73ada172130b |
| institution | Directory Open Access Journal |
| issn | 2561-326X |
| language | English |
| last_indexed | 2024-03-12T12:53:36Z |
| publishDate | 2022-05-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Formative Research |
| spelling | doaj.art-7d6ea91491484b1799ee73ada172130b2023-08-28T21:45:41ZengJMIR PublicationsJMIR Formative Research2561-326X2022-05-0165e3443610.2196/34436The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health RecordsVanessa Panaitehttps://orcid.org/0000-0003-4958-7992Andrew R Devendorfhttps://orcid.org/0000-0002-0142-6359Dezon Finchhttps://orcid.org/0000-0002-8241-5319Lina Bouayadhttps://orcid.org/0000-0001-9999-4989Stephen L Lutherhttps://orcid.org/0000-0001-7524-7380Susan K Schultzhttps://orcid.org/0000-0001-5846-8288 BackgroundAffective characteristics are associated with depression severity, course, and prognosis. Patients’ affect captured by clinicians during sessions may provide a rich source of information that more naturally aligns with the depression course and patient-desired depression outcomes. ObjectiveIn this paper, we propose an information extraction vocabulary used to pilot the feasibility and reliability of identifying clinician-recorded patient affective states in clinical notes from electronic health records. MethodsAffect and mood were annotated in 147 clinical notes of 109 patients by 2 independent coders across 3 pilots. Intercoder discrepancies were settled by a third coder. This reference annotation set was used to test a proof-of-concept natural language processing (NLP) system using a named entity recognition approach. ResultsConcepts were frequently addressed in templated format and free text in clinical notes. Annotated data demonstrated that affective characteristics were identified in 87.8% (129/147) of the notes, while mood was identified in 97.3% (143/147) of the notes. The intercoder reliability was consistently good across the pilots (interannotator agreement [IAA] >70%). The final NLP system showed good reliability with the final reference annotation set (mood IAA=85.8%; affect IAA=80.9%). ConclusionsAffect and mood can be reliably identified in clinician reports and are good targets for NLP. We discuss several next steps to expand on this proof of concept and the value of this research for depression clinical research.https://formative.jmir.org/2022/5/e34436 |
| spellingShingle | Vanessa Panaite Andrew R Devendorf Dezon Finch Lina Bouayad Stephen L Luther Susan K Schultz The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records JMIR Formative Research |
| title | The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records |
| title_full | The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records |
| title_fullStr | The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records |
| title_full_unstemmed | The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records |
| title_short | The Value of Extracting Clinician-Recorded Affect for Advancing Clinical Research on Depression: Proof-of-Concept Study Applying Natural Language Processing to Electronic Health Records |
| title_sort | value of extracting clinician recorded affect for advancing clinical research on depression proof of concept study applying natural language processing to electronic health records |
| url | https://formative.jmir.org/2022/5/e34436 |
| work_keys_str_mv | AT vanessapanaite thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT andrewrdevendorf thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT dezonfinch thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT linabouayad thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT stephenlluther thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT susankschultz thevalueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT vanessapanaite valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT andrewrdevendorf valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT dezonfinch valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT linabouayad valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT stephenlluther valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords AT susankschultz valueofextractingclinicianrecordedaffectforadvancingclinicalresearchondepressionproofofconceptstudyapplyingnaturallanguageprocessingtoelectronichealthrecords |