Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning
Abstract Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-02-01
|
| Series: | npj Schizophrenia |
| Online Access: | https://doi.org/10.1038/s41537-021-00197-6 |
| _version_ | 1797425113782550528 |
|---|---|
| author | Sagar Jilka Clarissa Mary Odoi Janet van Bilsen Daniel Morris Sinan Erturk Nicholas Cummins Matteo Cella Til Wykes |
| author_facet | Sagar Jilka Clarissa Mary Odoi Janet van Bilsen Daniel Morris Sinan Erturk Nicholas Cummins Matteo Cella Til Wykes |
| author_sort | Sagar Jilka |
| collection | DOAJ |
| description | Abstract Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success. |
| first_indexed | 2024-03-09T08:11:17Z |
| format | Article |
| id | doaj.art-2da3e705f405412b9dfa3eef9d5ef0c7 |
| institution | Directory Open Access Journal |
| issn | 2334-265X |
| language | English |
| last_indexed | 2024-03-09T08:11:17Z |
| publishDate | 2022-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Schizophrenia |
| spelling | doaj.art-2da3e705f405412b9dfa3eef9d5ef0c72023-12-02T23:17:11ZengNature Portfolionpj Schizophrenia2334-265X2022-02-01811810.1038/s41537-021-00197-6Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learningSagar Jilka0Clarissa Mary Odoi1Janet van Bilsen2Daniel Morris3Sinan Erturk4Nicholas Cummins5Matteo Cella6Til Wykes7Institute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonInstitute of Psychiatry, Psychology & Neuroscience, King’s College LondonAbstract Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.https://doi.org/10.1038/s41537-021-00197-6 |
| spellingShingle | Sagar Jilka Clarissa Mary Odoi Janet van Bilsen Daniel Morris Sinan Erturk Nicholas Cummins Matteo Cella Til Wykes Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning npj Schizophrenia |
| title | Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning |
| title_full | Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning |
| title_fullStr | Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning |
| title_full_unstemmed | Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning |
| title_short | Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning |
| title_sort | identifying schizophrenia stigma on twitter a proof of principle model using service user supervised machine learning |
| url | https://doi.org/10.1038/s41537-021-00197-6 |
| work_keys_str_mv | AT sagarjilka identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT clarissamaryodoi identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT janetvanbilsen identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT danielmorris identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT sinanerturk identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT nicholascummins identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT matteocella identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning AT tilwykes identifyingschizophreniastigmaontwitteraproofofprinciplemodelusingserviceusersupervisedmachinelearning |