Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study)
BackgroundArtificial intelligence (AI)–based chatbots could help address some of the challenges patients face in acquiring information essential to their self-health management, including unreliable sources and overburdened health care professionals. Research to ensure the pr...
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2024-02-01
|
| Series: | JMIR Research Protocols |
| Online Access: | https://www.researchprotocols.org/2024/1/e54668 |
| _version_ | 1797314078886068224 |
|---|---|
| author | Yuanchao Ma Sofiane Achiche Marie-Pascale Pomey Jesseca Paquette Nesrine Adjtoutah Serge Vicente Kim Engler Moustafa Laymouna David Lessard Benoît Lemire Jamil Asselah Rachel Therrien Esli Osmanlliu Ma'n H Zawati Yann Joly Bertrand Lebouché |
| author_facet | Yuanchao Ma Sofiane Achiche Marie-Pascale Pomey Jesseca Paquette Nesrine Adjtoutah Serge Vicente Kim Engler Moustafa Laymouna David Lessard Benoît Lemire Jamil Asselah Rachel Therrien Esli Osmanlliu Ma'n H Zawati Yann Joly Bertrand Lebouché |
| author_sort | Yuanchao Ma |
| collection | DOAJ |
| description |
BackgroundArtificial intelligence (AI)–based chatbots could help address some of the challenges patients face in acquiring information essential to their self-health management, including unreliable sources and overburdened health care professionals. Research to ensure the proper design, implementation, and uptake of chatbots is imperative. Inclusive digital health research and responsible AI integration into health care require active and sustained patient and stakeholder engagement, yet corresponding activities and guidance are limited for this purpose.
ObjectiveIn response, this manuscript presents a master protocol for the development, testing, and implementation of a chatbot family in partnership with stakeholders. This protocol aims to help efficiently translate an initial chatbot intervention (MARVIN) to multiple health domains and populations.
MethodsThe MARVIN chatbots study has an adaptive platform trial design consisting of multiple parallel individual chatbot substudies with four common objectives: (1) co-construct a tailored AI chatbot for a specific health care setting, (2) assess its usability with a small sample of participants, (3) measure implementation outcomes (usability, acceptability, appropriateness, adoption, and fidelity) within a large sample, and (4) evaluate the impact of patient and stakeholder partnerships on chatbot development. For objective 1, a needs assessment will be conducted within the setting, involving four 2-hour focus groups with 5 participants each. Then, a co-construction design committee will be formed with patient partners, health care professionals, and researchers who will participate in 6 workshops for chatbot development, testing, and improvement. For objective 2, a total of 30 participants will interact with the prototype for 3 weeks and assess its usability through a survey and 3 focus groups. Positive usability outcomes will lead to the initiation of objective 3, whereby the public will be able to access the chatbot for a 12-month real-world implementation study using web-based questionnaires to measure usability, acceptability, and appropriateness for 150 participants and meta-use data to inform adoption and fidelity. After each objective, for objective 4, focus groups will be conducted with the design committee to better understand their perspectives on the engagement process.
ResultsFrom July 2022 to October 2023, this master protocol led to four substudies conducted at the McGill University Health Centre or the Centre hospitalier de l’Université de Montréal (both in Montreal, Quebec, Canada): (1) MARVIN for HIV (large-scale implementation expected in mid-2024), (2) MARVIN-Pharma for community pharmacists providing HIV care (usability study planned for mid-2024), (3) MARVINA for breast cancer, and (4) MARVIN-CHAMP for pediatric infectious conditions (both in preparation, with development to begin in early 2024).
ConclusionsThis master protocol offers an approach to chatbot development in partnership with patients and health care professionals that includes a comprehensive assessment of implementation outcomes. It also contributes to best practice recommendations for patient and stakeholder engagement in digital health research.
Trial RegistrationClinicalTrials.gov NCT05789901; https://classic.clinicaltrials.gov/ct2/show/NCT05789901
International Registered Report Identifier (IRRID)PRR1-10.2196/54668 |
| first_indexed | 2024-03-08T02:40:53Z |
| format | Article |
| id | doaj.art-d34f4f8daea04cf790db3755d91eea35 |
| institution | Directory Open Access Journal |
| issn | 1929-0748 |
| language | English |
| last_indexed | 2024-03-08T02:40:53Z |
| publishDate | 2024-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Research Protocols |
| spelling | doaj.art-d34f4f8daea04cf790db3755d91eea352024-02-13T13:45:34ZengJMIR PublicationsJMIR Research Protocols1929-07482024-02-0113e5466810.2196/54668Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study)Yuanchao Mahttps://orcid.org/0000-0002-4048-1705Sofiane Achichehttps://orcid.org/0000-0002-7730-0701Marie-Pascale Pomeyhttps://orcid.org/0000-0001-5180-8139Jesseca Paquettehttps://orcid.org/0000-0003-0361-1440Nesrine Adjtoutahhttps://orcid.org/0009-0000-2751-9804Serge Vicentehttps://orcid.org/0000-0001-6478-4308Kim Englerhttps://orcid.org/0000-0001-8364-7421Moustafa Laymounahttps://orcid.org/0000-0002-0673-6356David Lessardhttps://orcid.org/0000-0002-1151-3763Benoît Lemirehttps://orcid.org/0000-0003-1363-7687Jamil Asselahhttps://orcid.org/0000-0002-6875-6512Rachel Therrienhttps://orcid.org/0000-0003-4170-017XEsli Osmanlliuhttps://orcid.org/0000-0001-5590-8866Ma'n H Zawatihttps://orcid.org/0000-0002-8905-6259Yann Jolyhttps://orcid.org/0000-0002-8775-2322Bertrand Lebouchéhttps://orcid.org/0000-0002-1273-9393 BackgroundArtificial intelligence (AI)–based chatbots could help address some of the challenges patients face in acquiring information essential to their self-health management, including unreliable sources and overburdened health care professionals. Research to ensure the proper design, implementation, and uptake of chatbots is imperative. Inclusive digital health research and responsible AI integration into health care require active and sustained patient and stakeholder engagement, yet corresponding activities and guidance are limited for this purpose. ObjectiveIn response, this manuscript presents a master protocol for the development, testing, and implementation of a chatbot family in partnership with stakeholders. This protocol aims to help efficiently translate an initial chatbot intervention (MARVIN) to multiple health domains and populations. MethodsThe MARVIN chatbots study has an adaptive platform trial design consisting of multiple parallel individual chatbot substudies with four common objectives: (1) co-construct a tailored AI chatbot for a specific health care setting, (2) assess its usability with a small sample of participants, (3) measure implementation outcomes (usability, acceptability, appropriateness, adoption, and fidelity) within a large sample, and (4) evaluate the impact of patient and stakeholder partnerships on chatbot development. For objective 1, a needs assessment will be conducted within the setting, involving four 2-hour focus groups with 5 participants each. Then, a co-construction design committee will be formed with patient partners, health care professionals, and researchers who will participate in 6 workshops for chatbot development, testing, and improvement. For objective 2, a total of 30 participants will interact with the prototype for 3 weeks and assess its usability through a survey and 3 focus groups. Positive usability outcomes will lead to the initiation of objective 3, whereby the public will be able to access the chatbot for a 12-month real-world implementation study using web-based questionnaires to measure usability, acceptability, and appropriateness for 150 participants and meta-use data to inform adoption and fidelity. After each objective, for objective 4, focus groups will be conducted with the design committee to better understand their perspectives on the engagement process. ResultsFrom July 2022 to October 2023, this master protocol led to four substudies conducted at the McGill University Health Centre or the Centre hospitalier de l’Université de Montréal (both in Montreal, Quebec, Canada): (1) MARVIN for HIV (large-scale implementation expected in mid-2024), (2) MARVIN-Pharma for community pharmacists providing HIV care (usability study planned for mid-2024), (3) MARVINA for breast cancer, and (4) MARVIN-CHAMP for pediatric infectious conditions (both in preparation, with development to begin in early 2024). ConclusionsThis master protocol offers an approach to chatbot development in partnership with patients and health care professionals that includes a comprehensive assessment of implementation outcomes. It also contributes to best practice recommendations for patient and stakeholder engagement in digital health research. Trial RegistrationClinicalTrials.gov NCT05789901; https://classic.clinicaltrials.gov/ct2/show/NCT05789901 International Registered Report Identifier (IRRID)PRR1-10.2196/54668https://www.researchprotocols.org/2024/1/e54668 |
| spellingShingle | Yuanchao Ma Sofiane Achiche Marie-Pascale Pomey Jesseca Paquette Nesrine Adjtoutah Serge Vicente Kim Engler Moustafa Laymouna David Lessard Benoît Lemire Jamil Asselah Rachel Therrien Esli Osmanlliu Ma'n H Zawati Yann Joly Bertrand Lebouché Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) JMIR Research Protocols |
| title | Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) |
| title_full | Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) |
| title_fullStr | Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) |
| title_full_unstemmed | Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) |
| title_short | Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study) |
| title_sort | adapting and evaluating an ai based chatbot through patient and stakeholder engagement to provide information for different health conditions master protocol for an adaptive platform trial the marvin chatbots study |
| url | https://www.researchprotocols.org/2024/1/e54668 |
| work_keys_str_mv | AT yuanchaoma adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT sofianeachiche adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT mariepascalepomey adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT jessecapaquette adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT nesrineadjtoutah adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT sergevicente adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT kimengler adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT moustafalaymouna adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT davidlessard adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT benoitlemire adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT jamilasselah adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT racheltherrien adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT esliosmanlliu adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT manhzawati adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT yannjoly adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy AT bertrandlebouche adaptingandevaluatinganaibasedchatbotthroughpatientandstakeholderengagementtoprovideinformationfordifferenthealthconditionsmasterprotocolforanadaptiveplatformtrialthemarvinchatbotsstudy |