Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study
Abstract Background For genetic conditions associated with neurodevelopmental disorder (GCAND), developmental domains such as motor ability, thinking and learning, social abilities, and communication are potential intervention targets. Performance on measures of developmental concepts can be express...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-01-01
|
| Series: | Journal of Neurodevelopmental Disorders |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s11689-022-09474-6 |
| _version_ | 1797946035098615808 |
|---|---|
| author | Cristan Farmer Audrey Thurm Jesse D. Troy Aaron J. Kaat |
| author_facet | Cristan Farmer Audrey Thurm Jesse D. Troy Aaron J. Kaat |
| author_sort | Cristan Farmer |
| collection | DOAJ |
| description | Abstract Background For genetic conditions associated with neurodevelopmental disorder (GCAND), developmental domains such as motor ability, thinking and learning, social abilities, and communication are potential intervention targets. Performance on measures of developmental concepts can be expressed using several types of scores. Norm-referenced scores are intended for the diagnostic context, allowing for the identification of impairment relative to age-based expectations, and can exhibit dramatic floor effects when used in individuals with more significant limitations. Person ability scores, which are derived via Rasch analysis or item response theory, are available on many standardized tests and are intended to measure within-person change. However, they have not been used or evaluated as primary endpoints in GCAND clinical trials. In this study, we simulated a series of parallel-arm clinical trials under several chronological age and impairment conditions, to compare empirically the power and type I error rate of operationalizing test performance using ability scores rather than norm-referenced scores. Results Using the Vineland Adaptive Behavior Scales as the example, we demonstrated an advantage in statistical power of ability scores over norm-referenced scores at extreme levels of impairment. This advantage was at least partially driven by floor effects in norm-referenced scores. For simulated conditions where impairment was less severe, ability scores outperformed norm-referenced scores, but they were more similar. The type I error rate closely approximated the nominal type I error rate of 5% for both scores. Conclusion The results of this simulation demonstrate a substantial power and interpretative advantage of ability scores over norm-referenced scores for studies of GCAND that will enroll participants with high levels of impairment. These results are expected to generalize to studies of developmental concepts, regardless of the etiology or specific test. However, the relative advantage of ability scores is expected to be even greater for tests with a higher floor than the Vineland. |
| first_indexed | 2024-04-10T21:05:35Z |
| format | Article |
| id | doaj.art-a3f91a454703464383d1979d9715624d |
| institution | Directory Open Access Journal |
| issn | 1866-1955 |
| language | English |
| last_indexed | 2024-04-10T21:05:35Z |
| publishDate | 2023-01-01 |
| publisher | BMC |
| record_format | Article |
| series | Journal of Neurodevelopmental Disorders |
| spelling | doaj.art-a3f91a454703464383d1979d9715624d2023-01-22T12:05:06ZengBMCJournal of Neurodevelopmental Disorders1866-19552023-01-011511910.1186/s11689-022-09474-6Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation studyCristan Farmer0Audrey Thurm1Jesse D. Troy2Aaron J. Kaat3Neurodevelopmental and Behavioral Phenotyping Service, National Institute of Mental HealthNeurodevelopmental and Behavioral Phenotyping Service, National Institute of Mental HealthDepartment of Biostatistics and Bioinformatics, Duke University School of MedicineFeinberg School of Medicine, Northwestern UniversityAbstract Background For genetic conditions associated with neurodevelopmental disorder (GCAND), developmental domains such as motor ability, thinking and learning, social abilities, and communication are potential intervention targets. Performance on measures of developmental concepts can be expressed using several types of scores. Norm-referenced scores are intended for the diagnostic context, allowing for the identification of impairment relative to age-based expectations, and can exhibit dramatic floor effects when used in individuals with more significant limitations. Person ability scores, which are derived via Rasch analysis or item response theory, are available on many standardized tests and are intended to measure within-person change. However, they have not been used or evaluated as primary endpoints in GCAND clinical trials. In this study, we simulated a series of parallel-arm clinical trials under several chronological age and impairment conditions, to compare empirically the power and type I error rate of operationalizing test performance using ability scores rather than norm-referenced scores. Results Using the Vineland Adaptive Behavior Scales as the example, we demonstrated an advantage in statistical power of ability scores over norm-referenced scores at extreme levels of impairment. This advantage was at least partially driven by floor effects in norm-referenced scores. For simulated conditions where impairment was less severe, ability scores outperformed norm-referenced scores, but they were more similar. The type I error rate closely approximated the nominal type I error rate of 5% for both scores. Conclusion The results of this simulation demonstrate a substantial power and interpretative advantage of ability scores over norm-referenced scores for studies of GCAND that will enroll participants with high levels of impairment. These results are expected to generalize to studies of developmental concepts, regardless of the etiology or specific test. However, the relative advantage of ability scores is expected to be even greater for tests with a higher floor than the Vineland.https://doi.org/10.1186/s11689-022-09474-6Ability scoreGrowth scale valueRasch analysisItem response theoryNeurodevelopmental disabilityRare genetic condition |
| spellingShingle | Cristan Farmer Audrey Thurm Jesse D. Troy Aaron J. Kaat Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study Journal of Neurodevelopmental Disorders Ability score Growth scale value Rasch analysis Item response theory Neurodevelopmental disability Rare genetic condition |
| title | Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study |
| title_full | Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study |
| title_fullStr | Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study |
| title_full_unstemmed | Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study |
| title_short | Comparing ability and norm-referenced scores as clinical trial outcomes for neurodevelopmental disabilities: a simulation study |
| title_sort | comparing ability and norm referenced scores as clinical trial outcomes for neurodevelopmental disabilities a simulation study |
| topic | Ability score Growth scale value Rasch analysis Item response theory Neurodevelopmental disability Rare genetic condition |
| url | https://doi.org/10.1186/s11689-022-09474-6 |
| work_keys_str_mv | AT cristanfarmer comparingabilityandnormreferencedscoresasclinicaltrialoutcomesforneurodevelopmentaldisabilitiesasimulationstudy AT audreythurm comparingabilityandnormreferencedscoresasclinicaltrialoutcomesforneurodevelopmentaldisabilitiesasimulationstudy AT jessedtroy comparingabilityandnormreferencedscoresasclinicaltrialoutcomesforneurodevelopmentaldisabilitiesasimulationstudy AT aaronjkaat comparingabilityandnormreferencedscoresasclinicaltrialoutcomesforneurodevelopmentaldisabilitiesasimulationstudy |