Pre-statistical harmonization of behavioral instruments across eight surveys and trials
Abstract Background Data harmonization is a powerful method to equilibrate items in measures that evaluate the same underlying construct. There are multiple measures to evaluate dementia related behavioral symptoms. Pre-statistical harmonization of behavioral instruments in dementia research is the...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2021-10-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-021-01431-6 |
| _version_ | 1797864391018807296 |
|---|---|
| author | Diefei Chen Eric Jutkowitz Skylar L. Iosepovici John C. Lin Alden L. Gross |
| author_facet | Diefei Chen Eric Jutkowitz Skylar L. Iosepovici John C. Lin Alden L. Gross |
| author_sort | Diefei Chen |
| collection | DOAJ |
| description | Abstract Background Data harmonization is a powerful method to equilibrate items in measures that evaluate the same underlying construct. There are multiple measures to evaluate dementia related behavioral symptoms. Pre-statistical harmonization of behavioral instruments in dementia research is the first step to develop a statistical crosswalk between measures. Studies that conduct pre-statistical harmonization of behavioral instruments rarely document their methods in a structured, reproducible manner. This is a crucial step which entails careful review, documentation and scrutiny of source data to ensure sufficient comparability between items prior to data pooling. Here, we document the pre-statistical harmonization of items measuring behavioral and psychological symptoms among people with dementia. We provide a box of recommended procedure for future studies. Methods We identified behavioral instruments that are used in clinical practice, a national survey, and randomized trials of dementia care interventions. We rigorously reviewed question content and scoring procedures to establish sufficient comparability across items as well as item quality prior to data pooling. Additionally, we standardized coding to Stata-readable format, which allowed us to automate approaches to identify potential cross-study differences in items and low-quality items. To ensure reasonable model fit for statistical co-calibration, we estimated two-parameter logistic Item Response Theory models within each of the eight studies. Results We identified 59 items from 11 behavioral instruments across the eight datasets. We found considerable cross-study heterogeneity in administration and coding procedures for items that measure the same attribute. Discrepancies existed in terms of directionality and quantification of behavioral symptoms for even seemingly comparable items. We resolved item response heterogeneity, missingness and skewness, conditional dependency prior to estimation of item response theory models for statistical co-calibration. We used several rigorous data transformation procedures to address these issues, including re-coding and truncation. Conclusions This study highlights the importance of each aspect involved in the pre-statistical harmonization process of behavioral instruments. We provide guidelines and recommendations for how future research may detect and account for similar issues in pooling behavioral and related instruments. |
| first_indexed | 2024-04-09T22:52:15Z |
| format | Article |
| id | doaj.art-e21ecedbd934455798464e7894dea385 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-09T22:52:15Z |
| publishDate | 2021-10-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-e21ecedbd934455798464e7894dea3852023-03-22T11:38:36ZengBMCBMC Medical Research Methodology1471-22882021-10-0121111110.1186/s12874-021-01431-6Pre-statistical harmonization of behavioral instruments across eight surveys and trialsDiefei Chen0Eric Jutkowitz1Skylar L. Iosepovici2John C. Lin3Alden L. Gross4Department of Epidemiology, Johns Hopkins Bloomberg School of Public HealthHealth Services, Policy & Practice, Brown School of Public HealthHealth Services, Policy & Practice, Brown School of Public HealthHealth Services, Policy & Practice, Brown School of Public HealthDepartment of Epidemiology, Johns Hopkins Bloomberg School of Public HealthAbstract Background Data harmonization is a powerful method to equilibrate items in measures that evaluate the same underlying construct. There are multiple measures to evaluate dementia related behavioral symptoms. Pre-statistical harmonization of behavioral instruments in dementia research is the first step to develop a statistical crosswalk between measures. Studies that conduct pre-statistical harmonization of behavioral instruments rarely document their methods in a structured, reproducible manner. This is a crucial step which entails careful review, documentation and scrutiny of source data to ensure sufficient comparability between items prior to data pooling. Here, we document the pre-statistical harmonization of items measuring behavioral and psychological symptoms among people with dementia. We provide a box of recommended procedure for future studies. Methods We identified behavioral instruments that are used in clinical practice, a national survey, and randomized trials of dementia care interventions. We rigorously reviewed question content and scoring procedures to establish sufficient comparability across items as well as item quality prior to data pooling. Additionally, we standardized coding to Stata-readable format, which allowed us to automate approaches to identify potential cross-study differences in items and low-quality items. To ensure reasonable model fit for statistical co-calibration, we estimated two-parameter logistic Item Response Theory models within each of the eight studies. Results We identified 59 items from 11 behavioral instruments across the eight datasets. We found considerable cross-study heterogeneity in administration and coding procedures for items that measure the same attribute. Discrepancies existed in terms of directionality and quantification of behavioral symptoms for even seemingly comparable items. We resolved item response heterogeneity, missingness and skewness, conditional dependency prior to estimation of item response theory models for statistical co-calibration. We used several rigorous data transformation procedures to address these issues, including re-coding and truncation. Conclusions This study highlights the importance of each aspect involved in the pre-statistical harmonization process of behavioral instruments. We provide guidelines and recommendations for how future research may detect and account for similar issues in pooling behavioral and related instruments.https://doi.org/10.1186/s12874-021-01431-6ADRDDementiaBehavioral symptom managementEpidemiologyPre-statistical harmonization |
| spellingShingle | Diefei Chen Eric Jutkowitz Skylar L. Iosepovici John C. Lin Alden L. Gross Pre-statistical harmonization of behavioral instruments across eight surveys and trials BMC Medical Research Methodology ADRD Dementia Behavioral symptom management Epidemiology Pre-statistical harmonization |
| title | Pre-statistical harmonization of behavioral instruments across eight surveys and trials |
| title_full | Pre-statistical harmonization of behavioral instruments across eight surveys and trials |
| title_fullStr | Pre-statistical harmonization of behavioral instruments across eight surveys and trials |
| title_full_unstemmed | Pre-statistical harmonization of behavioral instruments across eight surveys and trials |
| title_short | Pre-statistical harmonization of behavioral instruments across eight surveys and trials |
| title_sort | pre statistical harmonization of behavioral instruments across eight surveys and trials |
| topic | ADRD Dementia Behavioral symptom management Epidemiology Pre-statistical harmonization |
| url | https://doi.org/10.1186/s12874-021-01431-6 |
| work_keys_str_mv | AT diefeichen prestatisticalharmonizationofbehavioralinstrumentsacrosseightsurveysandtrials AT ericjutkowitz prestatisticalharmonizationofbehavioralinstrumentsacrosseightsurveysandtrials AT skylarliosepovici prestatisticalharmonizationofbehavioralinstrumentsacrosseightsurveysandtrials AT johnclin prestatisticalharmonizationofbehavioralinstrumentsacrosseightsurveysandtrials AT aldenlgross prestatisticalharmonizationofbehavioralinstrumentsacrosseightsurveysandtrials |