A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model
Background: A common way for computing test-retest reliability is Intra Class Correlation which was developed for continuous variables. But it widely used to assess test-retest reliability in questionnaires with Likert scales. Most of the time consecutive numbers regarded as option labels of a ques...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tehran University of Medical Sciences
2023-03-01
|
| Series: | Journal of Biostatistics and Epidemiology |
| Subjects: | |
| Online Access: | https://jbe.tums.ac.ir/index.php/jbe/article/view/865 |
| _version_ | 1827764733816602624 |
|---|---|
| author | Samira Chaibakhsh Asma Poorhosseingholi |
| author_facet | Samira Chaibakhsh Asma Poorhosseingholi |
| author_sort | Samira Chaibakhsh |
| collection | DOAJ |
| description |
Background: A common way for computing test-retest reliability is Intra Class Correlation which was developed for continuous variables. But it widely used to assess test-retest reliability in questionnaires with Likert scales. Most of the time consecutive numbers regarded as option labels of a question. If the probability of choosing options be the same, using this method is logic, otherwise it is not. Therefore, in this study a modified estimator of ICC is proposed to improve the estimation of ICC for ordinal scale by using latent variable model.
Method: In this method test-retest answers were considered as bivariate variables and cumulative Probit latent variable model was fitted. A simulation study with N=1500 replicates was conducted to compare the ICC estimations of Likert scale approach with a latent variable approach. Different sample sizes (n=20, 30) was generated with different correlation parameters. The simulations were repeated for questions with 3 and 5 options with different probability of selecting options of a question. After that the two approaches were run on Beck for suicidal ideation questionnaire.
Result: In general the difference between Likert scale approach and latent variable approach were higher in 3 question options compared to 5 and also by increasing sample size and correlation between bivariate data, Root Mean Square Errors and bias were decreased. Assuming different probabilities for options, there was a considerably difference between Root Mean Square Errors, bias and standard deviation of estimation of ICC in two models. Using latent variable approach resulted less bias, SD and Root Mean Square Errors especially in lower sample sizes.
Conclusion: Simulations showed when the probability of choosing options of a question are skewed, using this method reduced Root Mean Square Errors especially when the options are less. This method was affected more on standard deviation compare to bias of estimations.
|
| first_indexed | 2024-03-11T11:09:33Z |
| format | Article |
| id | doaj.art-83feaf66b6ad43d382c31aba3bc643e6 |
| institution | Directory Open Access Journal |
| issn | 2383-4196 2383-420X |
| language | English |
| last_indexed | 2024-03-11T11:09:33Z |
| publishDate | 2023-03-01 |
| publisher | Tehran University of Medical Sciences |
| record_format | Article |
| series | Journal of Biostatistics and Epidemiology |
| spelling | doaj.art-83feaf66b6ad43d382c31aba3bc643e62023-11-12T07:03:18ZengTehran University of Medical SciencesJournal of Biostatistics and Epidemiology2383-41962383-420X2023-03-019110.18502/jbe.v9i1.13972A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable ModelSamira Chaibakhsh0Asma Poorhosseingholi1SBMUIran University of Medical Scienses Background: A common way for computing test-retest reliability is Intra Class Correlation which was developed for continuous variables. But it widely used to assess test-retest reliability in questionnaires with Likert scales. Most of the time consecutive numbers regarded as option labels of a question. If the probability of choosing options be the same, using this method is logic, otherwise it is not. Therefore, in this study a modified estimator of ICC is proposed to improve the estimation of ICC for ordinal scale by using latent variable model. Method: In this method test-retest answers were considered as bivariate variables and cumulative Probit latent variable model was fitted. A simulation study with N=1500 replicates was conducted to compare the ICC estimations of Likert scale approach with a latent variable approach. Different sample sizes (n=20, 30) was generated with different correlation parameters. The simulations were repeated for questions with 3 and 5 options with different probability of selecting options of a question. After that the two approaches were run on Beck for suicidal ideation questionnaire. Result: In general the difference between Likert scale approach and latent variable approach were higher in 3 question options compared to 5 and also by increasing sample size and correlation between bivariate data, Root Mean Square Errors and bias were decreased. Assuming different probabilities for options, there was a considerably difference between Root Mean Square Errors, bias and standard deviation of estimation of ICC in two models. Using latent variable approach resulted less bias, SD and Root Mean Square Errors especially in lower sample sizes. Conclusion: Simulations showed when the probability of choosing options of a question are skewed, using this method reduced Root Mean Square Errors especially when the options are less. This method was affected more on standard deviation compare to bias of estimations. https://jbe.tums.ac.ir/index.php/jbe/article/view/865intra class correlationreliabilitylatent variablemultivariate modeltest-retest |
| spellingShingle | Samira Chaibakhsh Asma Poorhosseingholi A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model Journal of Biostatistics and Epidemiology intra class correlation reliability latent variable multivariate model test-retest |
| title | A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model |
| title_full | A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model |
| title_fullStr | A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model |
| title_full_unstemmed | A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model |
| title_short | A Modification on Intra Class Correlation Estimation for Ordinal Scale Variable Using Latent Variable Model |
| title_sort | modification on intra class correlation estimation for ordinal scale variable using latent variable model |
| topic | intra class correlation reliability latent variable multivariate model test-retest |
| url | https://jbe.tums.ac.ir/index.php/jbe/article/view/865 |
| work_keys_str_mv | AT samirachaibakhsh amodificationonintraclasscorrelationestimationforordinalscalevariableusinglatentvariablemodel AT asmapoorhosseingholi amodificationonintraclasscorrelationestimationforordinalscalevariableusinglatentvariablemodel AT samirachaibakhsh modificationonintraclasscorrelationestimationforordinalscalevariableusinglatentvariablemodel AT asmapoorhosseingholi modificationonintraclasscorrelationestimationforordinalscalevariableusinglatentvariablemodel |