A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study
BackgroundDepression is a major global cause of morbidity, an economic burden, and the greatest health challenge leading to chronic disability. Mobile monitoring of mental conditions has long been a sought-after metric to overcome the problems associated with the screening, d...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-05-01
|
| Series: | JMIR Formative Research |
| Online Access: | https://formative.jmir.org/2022/5/e37736 |
| _version_ | 1797735053838516224 |
|---|---|
| author | Soumya Choudhary Nikita Thomas Janine Ellenberger Girish Srinivasan Roy Cohen |
| author_facet | Soumya Choudhary Nikita Thomas Janine Ellenberger Girish Srinivasan Roy Cohen |
| author_sort | Soumya Choudhary |
| collection | DOAJ |
| description |
BackgroundDepression is a major global cause of morbidity, an economic burden, and the greatest health challenge leading to chronic disability. Mobile monitoring of mental conditions has long been a sought-after metric to overcome the problems associated with the screening, diagnosis, and monitoring of depression and its heterogeneous presentation. The widespread availability of smartphones has made it possible to use their data to generate digital behavioral models that can be used for both clinical and remote screening and monitoring purposes. This study is novel as it adds to the field by conducting a trial using private and nonintrusive sensors that can help detect and monitor depression in a continuous, passive manner.
ObjectiveThis study demonstrates a novel mental behavioral profiling metric (the Mental Health Similarity Score), derived from analyzing passively monitored, private, and nonintrusive smartphone use data, to identify and track depressive behavior and its progression.
MethodsSmartphone data sets and self-reported Patient Health Questionnaire-9 (PHQ-9) depression assessments were collected from 558 smartphone users on the Android operating system in an observational study over an average of 10.7 (SD 23.7) days. We quantified 37 digital behavioral markers from the passive smartphone data set and explored the relationship between the digital behavioral markers and depression using correlation coefficients and random forest models. We leveraged 4 supervised machine learning classification algorithms to predict depression and its severity using PHQ-9 scores as the ground truth. We also quantified an additional 3 digital markers from gyroscope sensors and explored their feasibility in improving the model’s accuracy in detecting depression.
ResultsThe PHQ-9 2-class model (none vs severe) achieved the following metrics: precision of 85% to 89%, recall of 85% to 89%, F1 of 87%, and accuracy of 87%. The PHQ-9 3-class model (none vs mild vs severe) achieved the following metrics: precision of 74% to 86%, recall of 76% to 83%, F1 of 75% to 84%, and accuracy of 78%. A significant positive Pearson correlation was found between PHQ-9 questions 2, 6, and 9 within the severely depressed users and the mental behavioral profiling metric (r=0.73). The PHQ-9 question-specific model achieved the following metrics: precision of 76% to 80%, recall of 75% to 81%, F1 of 78% to 89%, and accuracy of 78%. When a gyroscope sensor was added as a feature, the Pearson correlation among questions 2, 6, and 9 decreased from 0.73 to 0.46. The PHQ-9 2-class model+gyro features achieved the following metrics: precision of 74% to 78%, recall of 67% to 83%, F1 of 72% to 78%, and accuracy of 76%.
ConclusionsOur results demonstrate that the Mental Health Similarity Score can be used to identify and track depressive behavior and its progression with high accuracy. |
| first_indexed | 2024-03-12T12:53:27Z |
| format | Article |
| id | doaj.art-57a3daf5be2040f99b86df0d69ca8333 |
| institution | Directory Open Access Journal |
| issn | 2561-326X |
| language | English |
| last_indexed | 2024-03-12T12:53:27Z |
| publishDate | 2022-05-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Formative Research |
| spelling | doaj.art-57a3daf5be2040f99b86df0d69ca83332023-08-28T21:46:55ZengJMIR PublicationsJMIR Formative Research2561-326X2022-05-0165e3773610.2196/37736A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational StudySoumya Choudharyhttps://orcid.org/0000-0002-7522-6923Nikita Thomashttps://orcid.org/0000-0002-4731-1485Janine Ellenbergerhttps://orcid.org/0000-0002-5452-5879Girish Srinivasanhttps://orcid.org/0000-0003-1475-5538Roy Cohenhttps://orcid.org/0000-0003-1997-9688 BackgroundDepression is a major global cause of morbidity, an economic burden, and the greatest health challenge leading to chronic disability. Mobile monitoring of mental conditions has long been a sought-after metric to overcome the problems associated with the screening, diagnosis, and monitoring of depression and its heterogeneous presentation. The widespread availability of smartphones has made it possible to use their data to generate digital behavioral models that can be used for both clinical and remote screening and monitoring purposes. This study is novel as it adds to the field by conducting a trial using private and nonintrusive sensors that can help detect and monitor depression in a continuous, passive manner. ObjectiveThis study demonstrates a novel mental behavioral profiling metric (the Mental Health Similarity Score), derived from analyzing passively monitored, private, and nonintrusive smartphone use data, to identify and track depressive behavior and its progression. MethodsSmartphone data sets and self-reported Patient Health Questionnaire-9 (PHQ-9) depression assessments were collected from 558 smartphone users on the Android operating system in an observational study over an average of 10.7 (SD 23.7) days. We quantified 37 digital behavioral markers from the passive smartphone data set and explored the relationship between the digital behavioral markers and depression using correlation coefficients and random forest models. We leveraged 4 supervised machine learning classification algorithms to predict depression and its severity using PHQ-9 scores as the ground truth. We also quantified an additional 3 digital markers from gyroscope sensors and explored their feasibility in improving the model’s accuracy in detecting depression. ResultsThe PHQ-9 2-class model (none vs severe) achieved the following metrics: precision of 85% to 89%, recall of 85% to 89%, F1 of 87%, and accuracy of 87%. The PHQ-9 3-class model (none vs mild vs severe) achieved the following metrics: precision of 74% to 86%, recall of 76% to 83%, F1 of 75% to 84%, and accuracy of 78%. A significant positive Pearson correlation was found between PHQ-9 questions 2, 6, and 9 within the severely depressed users and the mental behavioral profiling metric (r=0.73). The PHQ-9 question-specific model achieved the following metrics: precision of 76% to 80%, recall of 75% to 81%, F1 of 78% to 89%, and accuracy of 78%. When a gyroscope sensor was added as a feature, the Pearson correlation among questions 2, 6, and 9 decreased from 0.73 to 0.46. The PHQ-9 2-class model+gyro features achieved the following metrics: precision of 74% to 78%, recall of 67% to 83%, F1 of 72% to 78%, and accuracy of 76%. ConclusionsOur results demonstrate that the Mental Health Similarity Score can be used to identify and track depressive behavior and its progression with high accuracy.https://formative.jmir.org/2022/5/e37736 |
| spellingShingle | Soumya Choudhary Nikita Thomas Janine Ellenberger Girish Srinivasan Roy Cohen A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study JMIR Formative Research |
| title | A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study |
| title_full | A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study |
| title_fullStr | A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study |
| title_full_unstemmed | A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study |
| title_short | A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study |
| title_sort | machine learning approach for detecting digital behavioral patterns of depression using nonintrusive smartphone data complementary path to patient health questionnaire 9 assessment prospective observational study |
| url | https://formative.jmir.org/2022/5/e37736 |
| work_keys_str_mv | AT soumyachoudhary amachinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT nikitathomas amachinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT janineellenberger amachinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT girishsrinivasan amachinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT roycohen amachinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT soumyachoudhary machinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT nikitathomas machinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT janineellenberger machinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT girishsrinivasan machinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy AT roycohen machinelearningapproachfordetectingdigitalbehavioralpatternsofdepressionusingnonintrusivesmartphonedatacomplementarypathtopatienthealthquestionnaire9assessmentprospectiveobservationalstudy |