Validating Biobehavioral Technologies for Use in Clinical Psychiatry
The last decade has witnessed the development of sophisticated biobehavioral and genetic, ambulatory, and other measures that promise unprecedented insight into psychiatric disorders. As yet, clinical sciences have struggled with implementing these objective measures and they have yet to move beyond...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-06-01
|
| Series: | Frontiers in Psychiatry |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyt.2021.503323/full |
| _version_ | 1830132449561018368 |
|---|---|
| author | Alex S. Cohen Alex S. Cohen Christopher R. Cox Raymond P. Tucker Kyle R. Mitchell Elana K. Schwartz Thanh P. Le Peter W. Foltz Terje B. Holmlund Brita Elvevåg Brita Elvevåg |
| author_facet | Alex S. Cohen Alex S. Cohen Christopher R. Cox Raymond P. Tucker Kyle R. Mitchell Elana K. Schwartz Thanh P. Le Peter W. Foltz Terje B. Holmlund Brita Elvevåg Brita Elvevåg |
| author_sort | Alex S. Cohen |
| collection | DOAJ |
| description | The last decade has witnessed the development of sophisticated biobehavioral and genetic, ambulatory, and other measures that promise unprecedented insight into psychiatric disorders. As yet, clinical sciences have struggled with implementing these objective measures and they have yet to move beyond “proof of concept.” In part, this struggle reflects a traditional, and conceptually flawed, application of traditional psychometrics (i.e., reliability and validity) for evaluating them. This paper focuses on “resolution,” concerning the degree to which changes in a signal can be detected and quantified, which is central to measurement evaluation in informatics, engineering, computational and biomedical sciences. We define and discuss resolution in terms of traditional reliability and validity evaluation for psychiatric measures, then highlight its importance in a study using acoustic features to predict self-injurious thoughts/behaviors (SITB). This study involved tracking natural language and self-reported symptoms in 124 psychiatric patients: (a) over 5–14 recording sessions, collected using a smart phone application, and (b) during a clinical interview. Importantly, the scope of these measures varied as a function of time (minutes, weeks) and spatial setting (i.e., smart phone vs. interview). Regarding reliability, acoustic features were temporally unstable until we specified the level of temporal/spatial resolution. Regarding validity, accuracy based on machine learning of acoustic features predicting SITB varied as a function of resolution. High accuracy was achieved (i.e., ~87%), but only when the acoustic and SITB measures were “temporally-matched” in resolution was the model generalizable to new data. Unlocking the potential of biobehavioral technologies for clinical psychiatry will require careful consideration of resolution. |
| first_indexed | 2024-12-17T06:35:32Z |
| format | Article |
| id | doaj.art-5c963f38c097488d87019feb05bb2fbe |
| institution | Directory Open Access Journal |
| issn | 1664-0640 |
| language | English |
| last_indexed | 2024-12-17T06:35:32Z |
| publishDate | 2021-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychiatry |
| spelling | doaj.art-5c963f38c097488d87019feb05bb2fbe2022-12-21T22:00:01ZengFrontiers Media S.A.Frontiers in Psychiatry1664-06402021-06-011210.3389/fpsyt.2021.503323503323Validating Biobehavioral Technologies for Use in Clinical PsychiatryAlex S. Cohen0Alex S. Cohen1Christopher R. Cox2Raymond P. Tucker3Kyle R. Mitchell4Elana K. Schwartz5Thanh P. Le6Peter W. Foltz7Terje B. Holmlund8Brita Elvevåg9Brita Elvevåg10Department of Psychology, Louisiana State University, Baton Rouge, LA, United StatesCenter for Computation and Technology Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, Louisiana State University, Baton Rouge, LA, United StatesDepartment of Psychology, University of Colorado, Boulder, CO, United StatesDepartment of Clinical Medicine, University of Tromsø—The Arctic University of Norway, Tromsø, NorwayDepartment of Clinical Medicine, University of Tromsø—The Arctic University of Norway, Tromsø, NorwayThe Norwegian Center for eHealth Research, University Hospital of North Norway, Tromsø, NorwayThe last decade has witnessed the development of sophisticated biobehavioral and genetic, ambulatory, and other measures that promise unprecedented insight into psychiatric disorders. As yet, clinical sciences have struggled with implementing these objective measures and they have yet to move beyond “proof of concept.” In part, this struggle reflects a traditional, and conceptually flawed, application of traditional psychometrics (i.e., reliability and validity) for evaluating them. This paper focuses on “resolution,” concerning the degree to which changes in a signal can be detected and quantified, which is central to measurement evaluation in informatics, engineering, computational and biomedical sciences. We define and discuss resolution in terms of traditional reliability and validity evaluation for psychiatric measures, then highlight its importance in a study using acoustic features to predict self-injurious thoughts/behaviors (SITB). This study involved tracking natural language and self-reported symptoms in 124 psychiatric patients: (a) over 5–14 recording sessions, collected using a smart phone application, and (b) during a clinical interview. Importantly, the scope of these measures varied as a function of time (minutes, weeks) and spatial setting (i.e., smart phone vs. interview). Regarding reliability, acoustic features were temporally unstable until we specified the level of temporal/spatial resolution. Regarding validity, accuracy based on machine learning of acoustic features predicting SITB varied as a function of resolution. High accuracy was achieved (i.e., ~87%), but only when the acoustic and SITB measures were “temporally-matched” in resolution was the model generalizable to new data. Unlocking the potential of biobehavioral technologies for clinical psychiatry will require careful consideration of resolution.https://www.frontiersin.org/articles/10.3389/fpsyt.2021.503323/fulldigital phenotypingserious mental illnessclinical sciencepsychiatric illnessbiobehavioralpsychometrics |
| spellingShingle | Alex S. Cohen Alex S. Cohen Christopher R. Cox Raymond P. Tucker Kyle R. Mitchell Elana K. Schwartz Thanh P. Le Peter W. Foltz Terje B. Holmlund Brita Elvevåg Brita Elvevåg Validating Biobehavioral Technologies for Use in Clinical Psychiatry Frontiers in Psychiatry digital phenotyping serious mental illness clinical science psychiatric illness biobehavioral psychometrics |
| title | Validating Biobehavioral Technologies for Use in Clinical Psychiatry |
| title_full | Validating Biobehavioral Technologies for Use in Clinical Psychiatry |
| title_fullStr | Validating Biobehavioral Technologies for Use in Clinical Psychiatry |
| title_full_unstemmed | Validating Biobehavioral Technologies for Use in Clinical Psychiatry |
| title_short | Validating Biobehavioral Technologies for Use in Clinical Psychiatry |
| title_sort | validating biobehavioral technologies for use in clinical psychiatry |
| topic | digital phenotyping serious mental illness clinical science psychiatric illness biobehavioral psychometrics |
| url | https://www.frontiersin.org/articles/10.3389/fpsyt.2021.503323/full |
| work_keys_str_mv | AT alexscohen validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT alexscohen validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT christopherrcox validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT raymondptucker validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT kylermitchell validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT elanakschwartz validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT thanhple validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT peterwfoltz validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT terjebholmlund validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT britaelvevag validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry AT britaelvevag validatingbiobehavioraltechnologiesforuseinclinicalpsychiatry |