Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index
Background:. This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI). Methods:. The study involved simulation using retrospectively collected real-world data. Previously developed C...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer
2023-03-01
|
| Series: | JBJS Open Access |
| Online Access: | http://journals.lww.com/jbjsoa/fulltext/10.2106/JBJS.OA.22.00036 |
| _version_ | 1828048181387067392 |
|---|---|
| author | Tracy Y. Zhu, PhD Otho R. Plummer, PhD Audrey Hunt, BS Alexander Joeris, MD, MSc |
| author_facet | Tracy Y. Zhu, PhD Otho R. Plummer, PhD Audrey Hunt, BS Alexander Joeris, MD, MSc |
| author_sort | Tracy Y. Zhu, PhD |
| collection | DOAJ |
| description | Background:. This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI).
Methods:. The study involved simulation using retrospectively collected real-world data. Previously developed CAT models of the ODI and NDI were applied to the responses from 52,551 and 18,196 patients with spinal conditions, respectively. Efficiency was evaluated by the reduction in the number of questions administered. Accuracy was evaluated by comparing means and standard deviations, calculating Pearson r and intraclass correlation coefficient (ICC) values, plotting the frequency distributions of CAT and full questionnaire scores, plotting the frequency distributions of differences between paired scores, and Bland-Altman plotting. Score changes, calculated as the postoperative ODI or NDI scores minus the preoperative scores, were compared between the CAT and full versions in patients for whom both preoperative and postoperative ODI or NDI questionnaires were available.
Results:. CAT models of the ODI and NDI required an average of 4.47 and 4.03 fewer questions per patient, respectively. The mean CAT ODI score was 0.7 point lower than the full ODI score (35.4 ± 19.0 versus 36.1 ± 19.3), and the mean CAT NDI score was 1.0 point lower than the full NDI score (34.7 ± 19.3 versus 33.8 ± 18.5). The Pearson r was 0.97 for both the ODI and NDI, and the ICC was 0.97 for both. The frequency distributions of the CAT and full scores showed marked overlap for the ODI and NDI. Differences between paired scores were less than the minimum clinically important difference in 98.9% of cases for the ODI and 98.5% for the NDI. Bland-Altman plots showed no proportional bias. The ODI and NDI score changes could be calculated in a subgroup of 6,044 and 4,775 patients, respectively; the distributions of the ODI and NDI score changes were near identical between the CAT and full versions.
Conclusions:. CAT models were able to reduce the question burden of the ODI and NDI. Scores obtained from the CAT models were faithful to those from the full questionnaires, both on the population level and on the individual patient level.
Level of Evidence:. Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence. |
| first_indexed | 2024-04-10T18:52:25Z |
| format | Article |
| id | doaj.art-589abdade78f49c4b6f12ae7b23d3e71 |
| institution | Directory Open Access Journal |
| issn | 2472-7245 |
| language | English |
| last_indexed | 2024-04-10T18:52:25Z |
| publishDate | 2023-03-01 |
| publisher | Wolters Kluwer |
| record_format | Article |
| series | JBJS Open Access |
| spelling | doaj.art-589abdade78f49c4b6f12ae7b23d3e712023-02-01T06:05:37ZengWolters KluwerJBJS Open Access2472-72452023-03-018110.2106/JBJS.OA.22.00036JBJSOA2200036Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability IndexTracy Y. Zhu, PhD0Otho R. Plummer, PhD1Audrey Hunt, BS2Alexander Joeris, MD, MSc31 Clinical Science, AO Innovation Translation Center, AO Foundation, Davos, Switzerland2 Universal Research Solutions, LLC, Columbia, Missouri2 Universal Research Solutions, LLC, Columbia, Missouri1 Clinical Science, AO Innovation Translation Center, AO Foundation, Davos, SwitzerlandBackground:. This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI). Methods:. The study involved simulation using retrospectively collected real-world data. Previously developed CAT models of the ODI and NDI were applied to the responses from 52,551 and 18,196 patients with spinal conditions, respectively. Efficiency was evaluated by the reduction in the number of questions administered. Accuracy was evaluated by comparing means and standard deviations, calculating Pearson r and intraclass correlation coefficient (ICC) values, plotting the frequency distributions of CAT and full questionnaire scores, plotting the frequency distributions of differences between paired scores, and Bland-Altman plotting. Score changes, calculated as the postoperative ODI or NDI scores minus the preoperative scores, were compared between the CAT and full versions in patients for whom both preoperative and postoperative ODI or NDI questionnaires were available. Results:. CAT models of the ODI and NDI required an average of 4.47 and 4.03 fewer questions per patient, respectively. The mean CAT ODI score was 0.7 point lower than the full ODI score (35.4 ± 19.0 versus 36.1 ± 19.3), and the mean CAT NDI score was 1.0 point lower than the full NDI score (34.7 ± 19.3 versus 33.8 ± 18.5). The Pearson r was 0.97 for both the ODI and NDI, and the ICC was 0.97 for both. The frequency distributions of the CAT and full scores showed marked overlap for the ODI and NDI. Differences between paired scores were less than the minimum clinically important difference in 98.9% of cases for the ODI and 98.5% for the NDI. Bland-Altman plots showed no proportional bias. The ODI and NDI score changes could be calculated in a subgroup of 6,044 and 4,775 patients, respectively; the distributions of the ODI and NDI score changes were near identical between the CAT and full versions. Conclusions:. CAT models were able to reduce the question burden of the ODI and NDI. Scores obtained from the CAT models were faithful to those from the full questionnaires, both on the population level and on the individual patient level. Level of Evidence:. Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.http://journals.lww.com/jbjsoa/fulltext/10.2106/JBJS.OA.22.00036 |
| spellingShingle | Tracy Y. Zhu, PhD Otho R. Plummer, PhD Audrey Hunt, BS Alexander Joeris, MD, MSc Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index JBJS Open Access |
| title | Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index |
| title_full | Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index |
| title_fullStr | Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index |
| title_full_unstemmed | Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index |
| title_short | Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index |
| title_sort | efficiency and accuracy of computerized adaptive testing for the oswestry disability index and neck disability index |
| url | http://journals.lww.com/jbjsoa/fulltext/10.2106/JBJS.OA.22.00036 |
| work_keys_str_mv | AT tracyyzhuphd efficiencyandaccuracyofcomputerizedadaptivetestingfortheoswestrydisabilityindexandneckdisabilityindex AT othorplummerphd efficiencyandaccuracyofcomputerizedadaptivetestingfortheoswestrydisabilityindexandneckdisabilityindex AT audreyhuntbs efficiencyandaccuracyofcomputerizedadaptivetestingfortheoswestrydisabilityindexandneckdisabilityindex AT alexanderjoerismdmsc efficiencyandaccuracyofcomputerizedadaptivetestingfortheoswestrydisabilityindexandneckdisabilityindex |