A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study
BackgroundApproximately 60%-80% of the primary care visits have a psychological stress component, but only 3% of patients receive stress management advice during these visits. Given recent advances in natural language processing, there is renewed interest in mental health cha...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-09-01
|
| Series: | JMIR Formative Research |
| Online Access: | https://formative.jmir.org/2021/9/e25294 |
| _version_ | 1797735716012163072 |
|---|---|
| author | Matthew Louis Mauriello Nantanick Tantivasadakarn Marco Antonio Mora-Mendoza Emmanuel Thierry Lincoln Grace Hon Parsa Nowruzi Dorien Simon Luke Hansen Nathaniel H Goenawan Joshua Kim Nikhil Gowda Dan Jurafsky Pablo Enrique Paredes |
| author_facet | Matthew Louis Mauriello Nantanick Tantivasadakarn Marco Antonio Mora-Mendoza Emmanuel Thierry Lincoln Grace Hon Parsa Nowruzi Dorien Simon Luke Hansen Nathaniel H Goenawan Joshua Kim Nikhil Gowda Dan Jurafsky Pablo Enrique Paredes |
| author_sort | Matthew Louis Mauriello |
| collection | DOAJ |
| description |
BackgroundApproximately 60%-80% of the primary care visits have a psychological stress component, but only 3% of patients receive stress management advice during these visits. Given recent advances in natural language processing, there is renewed interest in mental health chatbots. Conversational agents that can understand a user’s problems and deliver advice that mitigates the effects of daily stress could be an effective public health tool. However, such systems are complex to build and costly to develop.
ObjectiveTo address these challenges, our aim is to develop and evaluate a fully automated mobile suite of shallow chatbots—we call them Popbots—that may serve as a new species of chatbots and further complement human assistance in an ecosystem of stress management support.
MethodsAfter conducting an exploratory Wizard of Oz study (N=14) to evaluate the feasibility of a suite of multiple chatbots, we conducted a web-based study (N=47) to evaluate the implementation of our prototype. Each participant was randomly assigned to a different chatbot designed on the basis of a proven cognitive or behavioral intervention method. To measure the effectiveness of the chatbots, the participants’ stress levels were determined using self-reported psychometric evaluations (eg, web-based daily surveys and Patient Health Questionnaire-4). The participants in these studies were recruited through email and enrolled on the web, and some of them participated in follow-up interviews that were conducted in person or on the web (as necessary).
ResultsOf the 47 participants, 31 (66%) completed the main study. The findings suggest that the users viewed the conversations with our chatbots as helpful or at least neutral and came away with increasingly positive sentiment toward the use of chatbots for proactive stress management. Moreover, those users who used the system more often (ie, they had more than or equal to the median number of conversations) noted a decrease in depression symptoms compared with those who used the system less often based on a Wilcoxon signed-rank test (W=91.50; Z=−2.54; P=.01; r=0.47). The follow-up interviews with a subset of the participants indicated that half of the common daily stressors could be discussed with chatbots, potentially reducing the burden on human coping resources.
ConclusionsOur work suggests that suites of shallow chatbots may offer benefits for both users and designers. As a result, this study’s contributions include the design and evaluation of a novel suite of shallow chatbots for daily stress management, a summary of benefits and challenges associated with random delivery of multiple conversational interventions, and design guidelines and directions for future research into similar systems, including authoring chatbot systems and artificial intelligence–enabled recommendation algorithms. |
| first_indexed | 2024-03-12T13:03:05Z |
| format | Article |
| id | doaj.art-163adb4fee7b4221895762b46a3195f6 |
| institution | Directory Open Access Journal |
| issn | 2561-326X |
| language | English |
| last_indexed | 2024-03-12T13:03:05Z |
| publishDate | 2021-09-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Formative Research |
| spelling | doaj.art-163adb4fee7b4221895762b46a3195f62023-08-28T19:03:21ZengJMIR PublicationsJMIR Formative Research2561-326X2021-09-0159e2529410.2196/25294A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory StudyMatthew Louis Mauriellohttps://orcid.org/0000-0001-5359-6520Nantanick Tantivasadakarnhttps://orcid.org/0000-0002-9895-355XMarco Antonio Mora-Mendozahttps://orcid.org/0000-0002-7305-3520Emmanuel Thierry Lincolnhttps://orcid.org/0000-0002-4758-8345Grace Honhttps://orcid.org/0000-0001-9471-2209Parsa Nowruzihttps://orcid.org/0000-0001-5372-5513Dorien Simonhttps://orcid.org/0000-0002-4292-2447Luke Hansenhttps://orcid.org/0000-0002-9760-2563Nathaniel H Goenawanhttps://orcid.org/0000-0001-9989-7281Joshua Kimhttps://orcid.org/0000-0002-4576-419XNikhil Gowdahttps://orcid.org/0000-0001-7679-1556Dan Jurafskyhttps://orcid.org/0000-0002-6459-7745Pablo Enrique Paredeshttps://orcid.org/0000-0003-2431-9190 BackgroundApproximately 60%-80% of the primary care visits have a psychological stress component, but only 3% of patients receive stress management advice during these visits. Given recent advances in natural language processing, there is renewed interest in mental health chatbots. Conversational agents that can understand a user’s problems and deliver advice that mitigates the effects of daily stress could be an effective public health tool. However, such systems are complex to build and costly to develop. ObjectiveTo address these challenges, our aim is to develop and evaluate a fully automated mobile suite of shallow chatbots—we call them Popbots—that may serve as a new species of chatbots and further complement human assistance in an ecosystem of stress management support. MethodsAfter conducting an exploratory Wizard of Oz study (N=14) to evaluate the feasibility of a suite of multiple chatbots, we conducted a web-based study (N=47) to evaluate the implementation of our prototype. Each participant was randomly assigned to a different chatbot designed on the basis of a proven cognitive or behavioral intervention method. To measure the effectiveness of the chatbots, the participants’ stress levels were determined using self-reported psychometric evaluations (eg, web-based daily surveys and Patient Health Questionnaire-4). The participants in these studies were recruited through email and enrolled on the web, and some of them participated in follow-up interviews that were conducted in person or on the web (as necessary). ResultsOf the 47 participants, 31 (66%) completed the main study. The findings suggest that the users viewed the conversations with our chatbots as helpful or at least neutral and came away with increasingly positive sentiment toward the use of chatbots for proactive stress management. Moreover, those users who used the system more often (ie, they had more than or equal to the median number of conversations) noted a decrease in depression symptoms compared with those who used the system less often based on a Wilcoxon signed-rank test (W=91.50; Z=−2.54; P=.01; r=0.47). The follow-up interviews with a subset of the participants indicated that half of the common daily stressors could be discussed with chatbots, potentially reducing the burden on human coping resources. ConclusionsOur work suggests that suites of shallow chatbots may offer benefits for both users and designers. As a result, this study’s contributions include the design and evaluation of a novel suite of shallow chatbots for daily stress management, a summary of benefits and challenges associated with random delivery of multiple conversational interventions, and design guidelines and directions for future research into similar systems, including authoring chatbot systems and artificial intelligence–enabled recommendation algorithms.https://formative.jmir.org/2021/9/e25294 |
| spellingShingle | Matthew Louis Mauriello Nantanick Tantivasadakarn Marco Antonio Mora-Mendoza Emmanuel Thierry Lincoln Grace Hon Parsa Nowruzi Dorien Simon Luke Hansen Nathaniel H Goenawan Joshua Kim Nikhil Gowda Dan Jurafsky Pablo Enrique Paredes A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study JMIR Formative Research |
| title | A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study |
| title_full | A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study |
| title_fullStr | A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study |
| title_full_unstemmed | A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study |
| title_short | A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study |
| title_sort | suite of mobile conversational agents for daily stress management popbots mixed methods exploratory study |
| url | https://formative.jmir.org/2021/9/e25294 |
| work_keys_str_mv | AT matthewlouismauriello asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nantanicktantivasadakarn asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT marcoantoniomoramendoza asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT emmanuelthierrylincoln asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT gracehon asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT parsanowruzi asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT doriensimon asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT lukehansen asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nathanielhgoenawan asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT joshuakim asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nikhilgowda asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT danjurafsky asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT pabloenriqueparedes asuiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT matthewlouismauriello suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nantanicktantivasadakarn suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT marcoantoniomoramendoza suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT emmanuelthierrylincoln suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT gracehon suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT parsanowruzi suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT doriensimon suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT lukehansen suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nathanielhgoenawan suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT joshuakim suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT nikhilgowda suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT danjurafsky suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy AT pabloenriqueparedes suiteofmobileconversationalagentsfordailystressmanagementpopbotsmixedmethodsexploratorystudy |