From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models
Passively collected behavioral health data from ubiquitous sensors could provide mental health professionals valuable insights into patient's daily lives, but such efforts are impeded by disparate metrics, lack of interoperability, and unclear correlations between the measured signals and an in...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Association for Computing Machinery
2024
|
| Online Access: | https://hdl.handle.net/1721.1/155207 |
| _version_ | 1811088485004607488 |
|---|---|
| author | Englhardt, Zachary Ma, Chengqian Morris, Margaret E. Chang, Chun-Cheng Xu, Xuhai "Orson" Qin, Lianhui McDuff, Daniel Liu, Xin Patel, Shwetak Iyer, Vikram |
| author_facet | Englhardt, Zachary Ma, Chengqian Morris, Margaret E. Chang, Chun-Cheng Xu, Xuhai "Orson" Qin, Lianhui McDuff, Daniel Liu, Xin Patel, Shwetak Iyer, Vikram |
| author_sort | Englhardt, Zachary |
| collection | MIT |
| description | Passively collected behavioral health data from ubiquitous sensors could provide mental health professionals valuable insights into patient's daily lives, but such efforts are impeded by disparate metrics, lack of interoperability, and unclear correlations between the measured signals and an individual's mental health. To address these challenges, we pioneer the exploration of large language models (LLMs) to synthesize clinically relevant insights from multi-sensor data. We develop chain-of-thought prompting methods to generate LLM reasoning on how data pertaining to activity, sleep and social interaction relate to conditions such as depression and anxiety. We then prompt the LLM to perform binary classification, achieving accuracies of 61.1%, exceeding the state of the art. We find models like GPT-4 correctly reference numerical data 75% of the time.
While we began our investigation by developing methods to use LLMs to output binary classifications for conditions like depression, we find instead that their greatest potential value to clinicians lies not in diagnostic classification, but rather in rigorous analysis of diverse self-tracking data to generate natural language summaries that synthesize multiple data streams and identify potential concerns. Clinicians envisioned using these insights in a variety of ways, principally for fostering collaborative investigation with patients to strengthen the therapeutic alliance and guide treatment. We describe this collaborative engagement, additional envisioned uses, and associated concerns that must be addressed before adoption in real-world contexts. |
| first_indexed | 2024-09-23T14:02:54Z |
| format | Article |
| id | mit-1721.1/155207 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:02:54Z |
| publishDate | 2024 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | mit-1721.1/1552072024-09-20T04:30:30Z From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models Englhardt, Zachary Ma, Chengqian Morris, Margaret E. Chang, Chun-Cheng Xu, Xuhai "Orson" Qin, Lianhui McDuff, Daniel Liu, Xin Patel, Shwetak Iyer, Vikram Passively collected behavioral health data from ubiquitous sensors could provide mental health professionals valuable insights into patient's daily lives, but such efforts are impeded by disparate metrics, lack of interoperability, and unclear correlations between the measured signals and an individual's mental health. To address these challenges, we pioneer the exploration of large language models (LLMs) to synthesize clinically relevant insights from multi-sensor data. We develop chain-of-thought prompting methods to generate LLM reasoning on how data pertaining to activity, sleep and social interaction relate to conditions such as depression and anxiety. We then prompt the LLM to perform binary classification, achieving accuracies of 61.1%, exceeding the state of the art. We find models like GPT-4 correctly reference numerical data 75% of the time. While we began our investigation by developing methods to use LLMs to output binary classifications for conditions like depression, we find instead that their greatest potential value to clinicians lies not in diagnostic classification, but rather in rigorous analysis of diverse self-tracking data to generate natural language summaries that synthesize multiple data streams and identify potential concerns. Clinicians envisioned using these insights in a variety of ways, principally for fostering collaborative investigation with patients to strengthen the therapeutic alliance and guide treatment. We describe this collaborative engagement, additional envisioned uses, and associated concerns that must be addressed before adoption in real-world contexts. 2024-06-06T16:40:01Z 2024-06-06T16:40:01Z 2024-05-13 2024-06-01T07:58:35Z Article http://purl.org/eprint/type/JournalArticle 2474-9567 https://hdl.handle.net/1721.1/155207 Englhardt, Zachary, Ma, Chengqian, Morris, Margaret E., Chang, Chun-Cheng, Xu, Xuhai "Orson" et al. 2024. "From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8 (2). PUBLISHER_CC en 10.1145/3659604 Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ The author(s) application/pdf Association for Computing Machinery Association for Computing Machinery |
| spellingShingle | Englhardt, Zachary Ma, Chengqian Morris, Margaret E. Chang, Chun-Cheng Xu, Xuhai "Orson" Qin, Lianhui McDuff, Daniel Liu, Xin Patel, Shwetak Iyer, Vikram From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title | From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title_full | From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title_fullStr | From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title_full_unstemmed | From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title_short | From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models |
| title_sort | from classification to clinical insights towards analyzing and reasoning about mobile and behavioral health data with large language models |
| url | https://hdl.handle.net/1721.1/155207 |
| work_keys_str_mv | AT englhardtzachary fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT machengqian fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT morrismargarete fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT changchuncheng fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT xuxuhaiorson fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT qinlianhui fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT mcduffdaniel fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT liuxin fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT patelshwetak fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels AT iyervikram fromclassificationtoclinicalinsightstowardsanalyzingandreasoningaboutmobileandbehavioralhealthdatawithlargelanguagemodels |