Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study
Abstract Introduction Research focusing on cognitive aging and dementia is a global endeavor. However, cross‐national differences in cognition are embedded in other sociocultural differences, precluding direct comparisons of test scores. Such comparisons can be facilitated by co‐calibration using it...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-04-01
|
| Series: | Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/dad2.12438 |
| _version_ | 1797317306407190528 |
|---|---|
| author | Brandon E. Gavett Sindana D. Ilango Rebecca Koscik Yue Ma Benjamin Helfand Chloe W. Eng Alden Gross Emily H. Trittschuh Richard N. Jones Dan Mungas |
| author_facet | Brandon E. Gavett Sindana D. Ilango Rebecca Koscik Yue Ma Benjamin Helfand Chloe W. Eng Alden Gross Emily H. Trittschuh Richard N. Jones Dan Mungas |
| author_sort | Brandon E. Gavett |
| collection | DOAJ |
| description | Abstract Introduction Research focusing on cognitive aging and dementia is a global endeavor. However, cross‐national differences in cognition are embedded in other sociocultural differences, precluding direct comparisons of test scores. Such comparisons can be facilitated by co‐calibration using item response theory (IRT). The goal of this study was to explore, using simulation, the necessary conditions for accurate harmonization of cognitive data. Method Neuropsychological test scores from the US Health and Retirement Study (HRS) and the Mexican Health and Aging Study (MHAS) were subjected to IRT analysis to estimate item parameters and sample means and standard deviations. These estimates were used to generate simulated item response patterns under 10 scenarios that adjusted the quality and quantity of linking items used in harmonization. IRT‐derived factor scores were compared to the known population values to assess bias, efficiency, accuracy, and reliability of the harmonized data. Results The current configuration of HRS and MHAS data was not suitable for harmonization, as poor linking item quality led to large bias in both cohorts. Scenarios with more numerous and higher quality linking items led to less biased and more accurate harmonization. Discussion Linking items must possess low measurement error across the range of latent ability for co‐calibration to be successful. HIGHLIGHTS We developed a statistical simulation platform to evaluate the degree to which cross‐sample harmonization accuracy varies as a function of the quality and quantity of linking items. Two large studies of aging—one in Mexico and one in the United States—use three common items to measure cognition. These three common items have weak correspondence with the ability being measured and are all low in difficulty. Harmonized scores derived from the three common linking items will provide biased and inaccurate estimates of cognitive ability. Harmonization accuracy is greatest when linking items vary in difficulty and are strongly related to the ability being measured. |
| first_indexed | 2024-03-08T03:33:00Z |
| format | Article |
| id | doaj.art-330ebc768ddb4fed9996b7aa1b41c12b |
| institution | Directory Open Access Journal |
| issn | 2352-8729 |
| language | English |
| last_indexed | 2024-03-08T03:33:00Z |
| publishDate | 2023-04-01 |
| publisher | Wiley |
| record_format | Article |
| series | Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring |
| spelling | doaj.art-330ebc768ddb4fed9996b7aa1b41c12b2024-02-10T14:10:32ZengWileyAlzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring2352-87292023-04-01152n/an/a10.1002/dad2.12438Harmonization of cognitive screening tools for dementia across diverse samples: A simulation studyBrandon E. Gavett0Sindana D. Ilango1Rebecca Koscik2Yue Ma3Benjamin Helfand4Chloe W. Eng5Alden Gross6Emily H. Trittschuh7Richard N. Jones8Dan Mungas9School of Psychological Science University of Western Australia Perth Western Australia AustraliaDepartment of Epidemiology University of Washington School of Public Health Seattle Washington USAWisconsin Alzheimer's Institute University of Wisconsin School of Medicine and Public Health Madison Wisconsin USAWisconsin Alzheimer's Disease Research Center University of Wisconsin School of Medicine and Public Health Madison Wisconsin USADepartment of Emergency Medicine University of Massachusetts Medical School Worcester Massachusetts USADepartment of Epidemiology and Biostatistics University of California San Francisco San Francisco California USADepartment of Epidemiology Johns Hopkins Bloomberg School Public Health Baltimore Maryland USAVA Puget Sound Health Care System Geriatric Research Education and Clinical Care Seattle Washington USADepartments of Psychiatry and Human Behavior and Neurology Warren Alpert Medical School Brown University Providence Rhode Island USADepartment of Neurology University of California Sacramento California USAAbstract Introduction Research focusing on cognitive aging and dementia is a global endeavor. However, cross‐national differences in cognition are embedded in other sociocultural differences, precluding direct comparisons of test scores. Such comparisons can be facilitated by co‐calibration using item response theory (IRT). The goal of this study was to explore, using simulation, the necessary conditions for accurate harmonization of cognitive data. Method Neuropsychological test scores from the US Health and Retirement Study (HRS) and the Mexican Health and Aging Study (MHAS) were subjected to IRT analysis to estimate item parameters and sample means and standard deviations. These estimates were used to generate simulated item response patterns under 10 scenarios that adjusted the quality and quantity of linking items used in harmonization. IRT‐derived factor scores were compared to the known population values to assess bias, efficiency, accuracy, and reliability of the harmonized data. Results The current configuration of HRS and MHAS data was not suitable for harmonization, as poor linking item quality led to large bias in both cohorts. Scenarios with more numerous and higher quality linking items led to less biased and more accurate harmonization. Discussion Linking items must possess low measurement error across the range of latent ability for co‐calibration to be successful. HIGHLIGHTS We developed a statistical simulation platform to evaluate the degree to which cross‐sample harmonization accuracy varies as a function of the quality and quantity of linking items. Two large studies of aging—one in Mexico and one in the United States—use three common items to measure cognition. These three common items have weak correspondence with the ability being measured and are all low in difficulty. Harmonized scores derived from the three common linking items will provide biased and inaccurate estimates of cognitive ability. Harmonization accuracy is greatest when linking items vary in difficulty and are strongly related to the ability being measured.https://doi.org/10.1002/dad2.12438biascognitioncomputer simulationglobal healthitem response theory |
| spellingShingle | Brandon E. Gavett Sindana D. Ilango Rebecca Koscik Yue Ma Benjamin Helfand Chloe W. Eng Alden Gross Emily H. Trittschuh Richard N. Jones Dan Mungas Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring bias cognition computer simulation global health item response theory |
| title | Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study |
| title_full | Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study |
| title_fullStr | Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study |
| title_full_unstemmed | Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study |
| title_short | Harmonization of cognitive screening tools for dementia across diverse samples: A simulation study |
| title_sort | harmonization of cognitive screening tools for dementia across diverse samples a simulation study |
| topic | bias cognition computer simulation global health item response theory |
| url | https://doi.org/10.1002/dad2.12438 |
| work_keys_str_mv | AT brandonegavett harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT sindanadilango harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT rebeccakoscik harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT yuema harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT benjaminhelfand harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT chloeweng harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT aldengross harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT emilyhtrittschuh harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT richardnjones harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy AT danmungas harmonizationofcognitivescreeningtoolsfordementiaacrossdiversesamplesasimulationstudy |