Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis
BackgroundImpairments in speech production are a core symptom of non-affective psychosis (NAP). While traditional clinical ratings of patients’ speech involve a subjective human factor, modern methods of natural language processing (NLP) promise an automatic and objective way of analyzing patients’...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-07-01
|
| Series: | Frontiers in Psychiatry |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1208856/full |
| _version_ | 1797772998339461120 |
|---|---|
| author | Sandra Anna Just Anna-Lena Bröcker Galina Ryazanskaya Ivan Nenchev Maria Schneider Felix Bermpohl Andreas Heinz Christiane Montag |
| author_facet | Sandra Anna Just Anna-Lena Bröcker Galina Ryazanskaya Ivan Nenchev Maria Schneider Felix Bermpohl Andreas Heinz Christiane Montag |
| author_sort | Sandra Anna Just |
| collection | DOAJ |
| description | BackgroundImpairments in speech production are a core symptom of non-affective psychosis (NAP). While traditional clinical ratings of patients’ speech involve a subjective human factor, modern methods of natural language processing (NLP) promise an automatic and objective way of analyzing patients’ speech. This study aimed to validate NLP methods for analyzing speech production in NAP patients.MethodsSpeech samples from patients with a diagnosis of schizophrenia or schizoaffective disorder were obtained at two measurement points, 6 months apart. Out of N = 71 patients at T1, speech samples were also available for N = 54 patients at T2. Global and local models of semantic coherence as well as different word embeddings (word2vec vs. GloVe) were applied to the transcribed speech samples. They were tested and compared regarding their correlation with clinical ratings and external criteria from cross-sectional and longitudinal measurements.ResultsResults did not show differences for global vs. local coherence models and found more significant correlations between word2vec models and clinically relevant outcome variables than for GloVe models. Exploratory analysis of longitudinal data did not yield significant correlation with coherence scores.ConclusionThese results indicate that natural language processing methods need to be critically validated in more studies and carefully selected before clinical application. |
| first_indexed | 2024-03-12T21:59:52Z |
| format | Article |
| id | doaj.art-41bf9276b77040adaa65479f12dfc28e |
| institution | Directory Open Access Journal |
| issn | 1664-0640 |
| language | English |
| last_indexed | 2024-03-12T21:59:52Z |
| publishDate | 2023-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychiatry |
| spelling | doaj.art-41bf9276b77040adaa65479f12dfc28e2023-07-25T08:59:24ZengFrontiers Media S.A.Frontiers in Psychiatry1664-06402023-07-011410.3389/fpsyt.2023.12088561208856Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosisSandra Anna Just0Anna-Lena Bröcker1Galina Ryazanskaya2Ivan Nenchev3Maria Schneider4Felix Bermpohl5Andreas Heinz6Christiane Montag7Department of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyDepartment of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyDepartment of Linguistics, University of Potsdam, Potsdam, GermanyDepartment of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyIPB Institut für Integrative Psychotherapieausbildung Berlin, MSB Medical School Berlin, GmbH, Berlin, GermanyDepartment of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyDepartment of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyDepartment of Psychiatry and Neurosciences, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, GermanyBackgroundImpairments in speech production are a core symptom of non-affective psychosis (NAP). While traditional clinical ratings of patients’ speech involve a subjective human factor, modern methods of natural language processing (NLP) promise an automatic and objective way of analyzing patients’ speech. This study aimed to validate NLP methods for analyzing speech production in NAP patients.MethodsSpeech samples from patients with a diagnosis of schizophrenia or schizoaffective disorder were obtained at two measurement points, 6 months apart. Out of N = 71 patients at T1, speech samples were also available for N = 54 patients at T2. Global and local models of semantic coherence as well as different word embeddings (word2vec vs. GloVe) were applied to the transcribed speech samples. They were tested and compared regarding their correlation with clinical ratings and external criteria from cross-sectional and longitudinal measurements.ResultsResults did not show differences for global vs. local coherence models and found more significant correlations between word2vec models and clinically relevant outcome variables than for GloVe models. Exploratory analysis of longitudinal data did not yield significant correlation with coherence scores.ConclusionThese results indicate that natural language processing methods need to be critically validated in more studies and carefully selected before clinical application.https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1208856/fullcoherencespeech analysisautomated analysisnatural language processingartificial intelligencepsychosis |
| spellingShingle | Sandra Anna Just Anna-Lena Bröcker Galina Ryazanskaya Ivan Nenchev Maria Schneider Felix Bermpohl Andreas Heinz Christiane Montag Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis Frontiers in Psychiatry coherence speech analysis automated analysis natural language processing artificial intelligence psychosis |
| title | Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis |
| title_full | Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis |
| title_fullStr | Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis |
| title_full_unstemmed | Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis |
| title_short | Validation of natural language processing methods capturing semantic incoherence in the speech of patients with non-affective psychosis |
| title_sort | validation of natural language processing methods capturing semantic incoherence in the speech of patients with non affective psychosis |
| topic | coherence speech analysis automated analysis natural language processing artificial intelligence psychosis |
| url | https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1208856/full |
| work_keys_str_mv | AT sandraannajust validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT annalenabrocker validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT galinaryazanskaya validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT ivannenchev validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT mariaschneider validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT felixbermpohl validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT andreasheinz validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis AT christianemontag validationofnaturallanguageprocessingmethodscapturingsemanticincoherenceinthespeechofpatientswithnonaffectivepsychosis |