Recognizing schizophrenia using facial expressions based on convolutional neural network
Abstract Objective Facial expressions have been served as clinical symptoms to convey mental conditions in psychiatry. This paper proposes to recognize patients with schizophrenia (SCZ) using their facial images based on deep learning algorithm, and to investigate objective differences in facial exp...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-05-01
|
| Series: | Brain and Behavior |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/brb3.3002 |
| _version_ | 1797828942125596672 |
|---|---|
| author | Xiaofei Zhang Tongxin Li Conghui Wang Tian Tian Haizhu Pang Jisong Pang Chen Su Xiaomei Shi Jiangong Li Lina Ren Jing Wang Lulu Li Yanyan Ma Shen Li Lili Wang |
| author_facet | Xiaofei Zhang Tongxin Li Conghui Wang Tian Tian Haizhu Pang Jisong Pang Chen Su Xiaomei Shi Jiangong Li Lina Ren Jing Wang Lulu Li Yanyan Ma Shen Li Lili Wang |
| author_sort | Xiaofei Zhang |
| collection | DOAJ |
| description | Abstract Objective Facial expressions have been served as clinical symptoms to convey mental conditions in psychiatry. This paper proposes to recognize patients with schizophrenia (SCZ) using their facial images based on deep learning algorithm, and to investigate objective differences in facial expressions between SCZ patients and healthy controls using deep learning algorithm and statistical analyses. Methods The study consists of two parts. The first part recruited 106 SCZ patients and 101 healthy controls, and videotaped their facial expressions through a fixed experimental paradigm. The video data were randomly divided into two sets, one for training a convolutional neural network (CNN) with the classification of “healthy control” or “SCZ patient” as output and the other for evaluating the classification result of the trained CNN. In the second part, all facial images of the recruited participants were put into another CNN separately, which was priorly trained with a facial expression database and will output the most likely facial expressions of the recruited participants. Statistical analyses were performed on the obtained facial expressions to find out the objective differences in facial expressions between the two recruited groups. Results The trained CNN achieved an overall accuracy of 95.18% for classifying “healthy control” or “SCZ patient.” Statistical results on the obtained facial expressions demonstrated that the objective differences between the two recruited groups were statistically significant (p < .05). Conclusions Facial expressions hold great promise as SCZ clues with the help of deep learning algorithm. The proposed approach would be potentially applied to mobile devices for autorecognizing SCZ in the context of clinical and daily life. |
| first_indexed | 2024-04-09T13:12:29Z |
| format | Article |
| id | doaj.art-82ef123963d2428b9465309120acedb4 |
| institution | Directory Open Access Journal |
| issn | 2162-3279 |
| language | English |
| last_indexed | 2024-04-09T13:12:29Z |
| publishDate | 2023-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | Brain and Behavior |
| spelling | doaj.art-82ef123963d2428b9465309120acedb42023-05-12T06:32:35ZengWileyBrain and Behavior2162-32792023-05-01135n/an/a10.1002/brb3.3002Recognizing schizophrenia using facial expressions based on convolutional neural networkXiaofei Zhang0Tongxin Li1Conghui Wang2Tian Tian3Haizhu Pang4Jisong Pang5Chen Su6Xiaomei Shi7Jiangong Li8Lina Ren9Jing Wang10Lulu Li11Yanyan Ma12Shen Li13Lili Wang14Department of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaInstitute of Mental Health Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaInstitute of Mental Health Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaDepartment of Psychiatry Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University Tianjin ChinaAbstract Objective Facial expressions have been served as clinical symptoms to convey mental conditions in psychiatry. This paper proposes to recognize patients with schizophrenia (SCZ) using their facial images based on deep learning algorithm, and to investigate objective differences in facial expressions between SCZ patients and healthy controls using deep learning algorithm and statistical analyses. Methods The study consists of two parts. The first part recruited 106 SCZ patients and 101 healthy controls, and videotaped their facial expressions through a fixed experimental paradigm. The video data were randomly divided into two sets, one for training a convolutional neural network (CNN) with the classification of “healthy control” or “SCZ patient” as output and the other for evaluating the classification result of the trained CNN. In the second part, all facial images of the recruited participants were put into another CNN separately, which was priorly trained with a facial expression database and will output the most likely facial expressions of the recruited participants. Statistical analyses were performed on the obtained facial expressions to find out the objective differences in facial expressions between the two recruited groups. Results The trained CNN achieved an overall accuracy of 95.18% for classifying “healthy control” or “SCZ patient.” Statistical results on the obtained facial expressions demonstrated that the objective differences between the two recruited groups were statistically significant (p < .05). Conclusions Facial expressions hold great promise as SCZ clues with the help of deep learning algorithm. The proposed approach would be potentially applied to mobile devices for autorecognizing SCZ in the context of clinical and daily life.https://doi.org/10.1002/brb3.3002clinical cluesconvolutional neural networkfacial expressionsschizophrenia |
| spellingShingle | Xiaofei Zhang Tongxin Li Conghui Wang Tian Tian Haizhu Pang Jisong Pang Chen Su Xiaomei Shi Jiangong Li Lina Ren Jing Wang Lulu Li Yanyan Ma Shen Li Lili Wang Recognizing schizophrenia using facial expressions based on convolutional neural network Brain and Behavior clinical clues convolutional neural network facial expressions schizophrenia |
| title | Recognizing schizophrenia using facial expressions based on convolutional neural network |
| title_full | Recognizing schizophrenia using facial expressions based on convolutional neural network |
| title_fullStr | Recognizing schizophrenia using facial expressions based on convolutional neural network |
| title_full_unstemmed | Recognizing schizophrenia using facial expressions based on convolutional neural network |
| title_short | Recognizing schizophrenia using facial expressions based on convolutional neural network |
| title_sort | recognizing schizophrenia using facial expressions based on convolutional neural network |
| topic | clinical clues convolutional neural network facial expressions schizophrenia |
| url | https://doi.org/10.1002/brb3.3002 |
| work_keys_str_mv | AT xiaofeizhang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT tongxinli recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT conghuiwang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT tiantian recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT haizhupang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT jisongpang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT chensu recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT xiaomeishi recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT jiangongli recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT linaren recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT jingwang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT lululi recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT yanyanma recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT shenli recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork AT liliwang recognizingschizophreniausingfacialexpressionsbasedonconvolutionalneuralnetwork |