What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks
Recent evidence have demonstrated that facial expressions could be a valid and important aspect for depression recognition. Although various works have been achieved in automatic depression recognition, it is a challenge to explore the inherent nuances of facial expressions that might reveal the und...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10218351/ |
| _version_ | 1797692173371572224 |
|---|---|
| author | Yu Ma Jian Shen Zeguang Zhao Huajian Liang Yang Tan Zhenyu Liu Kun Qian Minqiang Yang Bin Hu |
| author_facet | Yu Ma Jian Shen Zeguang Zhao Huajian Liang Yang Tan Zhenyu Liu Kun Qian Minqiang Yang Bin Hu |
| author_sort | Yu Ma |
| collection | DOAJ |
| description | Recent evidence have demonstrated that facial expressions could be a valid and important aspect for depression recognition. Although various works have been achieved in automatic depression recognition, it is a challenge to explore the inherent nuances of facial expressions that might reveal the underlying differences between depressed patients and healthy subjects under different stimuli. There is a lack of an undisturbed system that monitors depressive patients’ mental states in various free-living scenarios, so this paper steps towards building a classification model where data collection, feature extraction, depression recognition and facial actions analysis are conducted to infer the differences of facial movements between depressive patients and healthy subjects. In this study, we firstly present a plan of dividing facial regions of interest to extract optical flow features of facial expressions for depression recognition. We then propose facial movements coefficients utilising discrete wavelet transformation. Specifically, Bayesian Networks equipped with construction of Pearson Correlation Coefficients based on discrete wavelet transformation is learnt, which allows for analysing movements of different facial regions. We evaluate our method on a clinically validated dataset of 30 depressed patients and 30 healthy control subjects, and experiments results obtained the accuracy and recall of 81.7%, 96.7%, respectively, outperforming other features for comparison. Most importantly, the Bayesian Networks we built on the coefficients under different stimuli may reveal some facial action patterns of depressed subjects, which have a potential to assist the automatic diagnosis of depression. |
| first_indexed | 2024-03-12T02:23:49Z |
| format | Article |
| id | doaj.art-1d6e38cbd18244689ded8311aafb9ea2 |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-12T02:23:49Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-1d6e38cbd18244689ded8311aafb9ea22023-09-05T23:00:14ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01313459346810.1109/TNSRE.2023.330535110218351What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian NetworksYu Ma0https://orcid.org/0000-0002-2235-2176Jian Shen1https://orcid.org/0000-0001-6099-3209Zeguang Zhao2Huajian Liang3Yang Tan4https://orcid.org/0000-0002-8188-0489Zhenyu Liu5https://orcid.org/0000-0001-8401-9056Kun Qian6https://orcid.org/0000-0002-1918-6453Minqiang Yang7https://orcid.org/0000-0002-7571-6439Bin Hu8https://orcid.org/0000-0003-3514-5413School of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Information Science and Engineering, Lanzhou University, Lanzhou, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Medical Technology, Beijing Institute of Technology, Beijing, ChinaSchool of Information Science and Engineering, Lanzhou University, Lanzhou, ChinaRecent evidence have demonstrated that facial expressions could be a valid and important aspect for depression recognition. Although various works have been achieved in automatic depression recognition, it is a challenge to explore the inherent nuances of facial expressions that might reveal the underlying differences between depressed patients and healthy subjects under different stimuli. There is a lack of an undisturbed system that monitors depressive patients’ mental states in various free-living scenarios, so this paper steps towards building a classification model where data collection, feature extraction, depression recognition and facial actions analysis are conducted to infer the differences of facial movements between depressive patients and healthy subjects. In this study, we firstly present a plan of dividing facial regions of interest to extract optical flow features of facial expressions for depression recognition. We then propose facial movements coefficients utilising discrete wavelet transformation. Specifically, Bayesian Networks equipped with construction of Pearson Correlation Coefficients based on discrete wavelet transformation is learnt, which allows for analysing movements of different facial regions. We evaluate our method on a clinically validated dataset of 30 depressed patients and 30 healthy control subjects, and experiments results obtained the accuracy and recall of 81.7%, 96.7%, respectively, outperforming other features for comparison. Most importantly, the Bayesian Networks we built on the coefficients under different stimuli may reveal some facial action patterns of depressed subjects, which have a potential to assist the automatic diagnosis of depression.https://ieeexplore.ieee.org/document/10218351/Depression recognitionfacial expressionsaction unitsoptical flowBayesian networks |
| spellingShingle | Yu Ma Jian Shen Zeguang Zhao Huajian Liang Yang Tan Zhenyu Liu Kun Qian Minqiang Yang Bin Hu What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks IEEE Transactions on Neural Systems and Rehabilitation Engineering Depression recognition facial expressions action units optical flow Bayesian networks |
| title | What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks |
| title_full | What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks |
| title_fullStr | What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks |
| title_full_unstemmed | What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks |
| title_short | What Can Facial Movements Reveal? Depression Recognition and Analysis Based on Optical Flow Using Bayesian Networks |
| title_sort | what can facial movements reveal depression recognition and analysis based on optical flow using bayesian networks |
| topic | Depression recognition facial expressions action units optical flow Bayesian networks |
| url | https://ieeexplore.ieee.org/document/10218351/ |
| work_keys_str_mv | AT yuma whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT jianshen whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT zeguangzhao whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT huajianliang whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT yangtan whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT zhenyuliu whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT kunqian whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT minqiangyang whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks AT binhu whatcanfacialmovementsrevealdepressionrecognitionandanalysisbasedonopticalflowusingbayesiannetworks |