The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery
BackgroundRoutine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-04-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2023/1/e41870 |
| _version_ | 1797734089115041792 |
|---|---|
| author | Conrad Harrison Inge Apon Kenny Ardouin Chris Sidey-Gibbons Anne Klassen Stefan Cano Karen Wong Riff Andrea Pusic Sarah Versnel Maarten Koudstaal Alexander C Allori Carolyn Rogers-Vizena Marc C Swan Dominic Furniss Jeremy Rodrigues |
| author_facet | Conrad Harrison Inge Apon Kenny Ardouin Chris Sidey-Gibbons Anne Klassen Stefan Cano Karen Wong Riff Andrea Pusic Sarah Versnel Maarten Koudstaal Alexander C Allori Carolyn Rogers-Vizena Marc C Swan Dominic Furniss Jeremy Rodrigues |
| author_sort | Conrad Harrison |
| collection | DOAJ |
| description |
BackgroundRoutine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant score interpretation. Recently, a PROM called the CLEFT-Q has been developed for use in the treatment of cleft lip or palate (CL/P), but the assessment burden may be limiting its uptake into clinical practice.
ObjectiveWe aimed to develop a CAT for the CLEFT-Q, which could facilitate the uptake of the CLEFT-Q PROM internationally. We aimed to conduct this work with a novel patient-centered approach and make source code available as an open-source framework for CAT development in other surgical conditions.
MethodsCATs were developed with the Rasch measurement theory, using full-length CLEFT-Q responses collected during the CLEFT-Q field test (this included 2434 patients across 12 countries). These algorithms were validated in Monte Carlo simulations involving full-length CLEFT-Q responses collected from 536 patients. In these simulations, the CAT algorithms approximated full-length CLEFT-Q scores iteratively, using progressively fewer items from the full-length PROM. Agreement between full-length CLEFT-Q score and CAT score at different assessment lengths was measured using the Pearson correlation coefficient, root-mean-square error (RMSE), and 95% limits of agreement. CAT settings, including the number of items to be included in the final assessments, were determined in a multistakeholder workshop that included patients and health care professionals. A user interface was developed for the platform, and it was prospectively piloted in the United Kingdom and the Netherlands. Interviews were conducted with 6 patients and 4 clinicians to explore end-user experience.
ResultsThe length of all 8 CLEFT-Q scales in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set combined was reduced from 76 to 59 items, and at this length, CAT assessments reproduced full-length CLEFT-Q scores accurately (with correlations between full-length CLEFT-Q score and CAT score exceeding 0.97, and the RMSE ranging from 2 to 5 out of 100). Workshop stakeholders considered this the optimal balance between accuracy and assessment burden. The platform was perceived to improve clinical communication and facilitate shared decision-making.
ConclusionsOur platform is likely to facilitate routine CLEFT-Q uptake, and this may have a positive impact on clinical care. Our free source code enables other researchers to rapidly and economically reproduce this work for other PROMs. |
| first_indexed | 2024-03-12T12:39:13Z |
| format | Article |
| id | doaj.art-4627ce5966804676afdefc052be49068 |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T12:39:13Z |
| publishDate | 2023-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-4627ce5966804676afdefc052be490682023-08-28T23:58:31ZengJMIR PublicationsJournal of Medical Internet Research1438-88712023-04-0125e4187010.2196/41870The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic SurgeryConrad Harrisonhttps://orcid.org/0000-0002-1428-5751Inge Aponhttps://orcid.org/0000-0002-7323-2676Kenny Ardouinhttps://orcid.org/0000-0002-8105-3926Chris Sidey-Gibbonshttps://orcid.org/0000-0002-4732-7305Anne Klassenhttps://orcid.org/0000-0003-4720-0096Stefan Canohttps://orcid.org/0000-0001-9390-273XKaren Wong Riffhttps://orcid.org/0000-0003-2031-7235Andrea Pusichttps://orcid.org/0000-0003-4352-9409Sarah Versnelhttps://orcid.org/0000-0002-0507-6106Maarten Koudstaalhttps://orcid.org/0000-0002-5060-7711Alexander C Allorihttps://orcid.org/0000-0003-3334-9877Carolyn Rogers-Vizenahttps://orcid.org/0000-0002-2166-1209Marc C Swanhttps://orcid.org/0000-0002-3110-5927Dominic Furnisshttps://orcid.org/0000-0003-2780-7173Jeremy Rodrigueshttps://orcid.org/0000-0002-9347-5026 BackgroundRoutine use of patient-reported outcome measures (PROMs) and computerized adaptive tests (CATs) may improve care in a range of surgical conditions. However, most available CATs are neither condition-specific nor coproduced with patients and lack clinically relevant score interpretation. Recently, a PROM called the CLEFT-Q has been developed for use in the treatment of cleft lip or palate (CL/P), but the assessment burden may be limiting its uptake into clinical practice. ObjectiveWe aimed to develop a CAT for the CLEFT-Q, which could facilitate the uptake of the CLEFT-Q PROM internationally. We aimed to conduct this work with a novel patient-centered approach and make source code available as an open-source framework for CAT development in other surgical conditions. MethodsCATs were developed with the Rasch measurement theory, using full-length CLEFT-Q responses collected during the CLEFT-Q field test (this included 2434 patients across 12 countries). These algorithms were validated in Monte Carlo simulations involving full-length CLEFT-Q responses collected from 536 patients. In these simulations, the CAT algorithms approximated full-length CLEFT-Q scores iteratively, using progressively fewer items from the full-length PROM. Agreement between full-length CLEFT-Q score and CAT score at different assessment lengths was measured using the Pearson correlation coefficient, root-mean-square error (RMSE), and 95% limits of agreement. CAT settings, including the number of items to be included in the final assessments, were determined in a multistakeholder workshop that included patients and health care professionals. A user interface was developed for the platform, and it was prospectively piloted in the United Kingdom and the Netherlands. Interviews were conducted with 6 patients and 4 clinicians to explore end-user experience. ResultsThe length of all 8 CLEFT-Q scales in the International Consortium for Health Outcomes Measurement (ICHOM) Standard Set combined was reduced from 76 to 59 items, and at this length, CAT assessments reproduced full-length CLEFT-Q scores accurately (with correlations between full-length CLEFT-Q score and CAT score exceeding 0.97, and the RMSE ranging from 2 to 5 out of 100). Workshop stakeholders considered this the optimal balance between accuracy and assessment burden. The platform was perceived to improve clinical communication and facilitate shared decision-making. ConclusionsOur platform is likely to facilitate routine CLEFT-Q uptake, and this may have a positive impact on clinical care. Our free source code enables other researchers to rapidly and economically reproduce this work for other PROMs.https://www.jmir.org/2023/1/e41870 |
| spellingShingle | Conrad Harrison Inge Apon Kenny Ardouin Chris Sidey-Gibbons Anne Klassen Stefan Cano Karen Wong Riff Andrea Pusic Sarah Versnel Maarten Koudstaal Alexander C Allori Carolyn Rogers-Vizena Marc C Swan Dominic Furniss Jeremy Rodrigues The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery Journal of Medical Internet Research |
| title | The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery |
| title_full | The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery |
| title_fullStr | The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery |
| title_full_unstemmed | The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery |
| title_short | The Development, Deployment, and Evaluation of the CLEFT-Q Computerized Adaptive Test: A Multimethods Approach Contributing to Personalized, Person-Centered Health Assessments in Plastic Surgery |
| title_sort | development deployment and evaluation of the cleft q computerized adaptive test a multimethods approach contributing to personalized person centered health assessments in plastic surgery |
| url | https://www.jmir.org/2023/1/e41870 |
| work_keys_str_mv | AT conradharrison thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT ingeapon thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT kennyardouin thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT chrissideygibbons thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT anneklassen thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT stefancano thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT karenwongriff thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT andreapusic thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT sarahversnel thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT maartenkoudstaal thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT alexandercallori thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT carolynrogersvizena thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT marccswan thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT dominicfurniss thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT jeremyrodrigues thedevelopmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT conradharrison developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT ingeapon developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT kennyardouin developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT chrissideygibbons developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT anneklassen developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT stefancano developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT karenwongriff developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT andreapusic developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT sarahversnel developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT maartenkoudstaal developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT alexandercallori developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT carolynrogersvizena developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT marccswan developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT dominicfurniss developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery AT jeremyrodrigues developmentdeploymentandevaluationofthecleftqcomputerizedadaptivetestamultimethodsapproachcontributingtopersonalizedpersoncenteredhealthassessmentsinplasticsurgery |