Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles
Cliff’s Delta is a non-parametric effect size that is based on data observations. In this paper, we introduce an adaptation of Cliff’s Delta in order to compare behaviour profiles. Behaviour profiles are density distributions in which survey answer behaviour is summarized for specific groups of res...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
European Survey Research Association
2022-12-01
|
| Series: | Survey Research Methods |
| Subjects: | |
| Online Access: | https://ojs.ub.uni-konstanz.de/srm/article/view/7908 |
| _version_ | 1797660684943622144 |
|---|---|
| author | Frank Bais Joost van der Neut |
| author_facet | Frank Bais Joost van der Neut |
| author_sort | Frank Bais |
| collection | DOAJ |
| description |
Cliff’s Delta is a non-parametric effect size that is based on data observations. In this paper, we introduce an adaptation of Cliff’s Delta in order to compare behaviour profiles. Behaviour profiles are density distributions in which survey answer behaviour is summarized for specific groups of respondents or items. Such profiles are useful, as they take into account the varying number of survey items that is filled out per respondent due to filter questions. By the adapted Cliff’s Delta, two subgroups of respondents (for instance higher and lower educated respondents) can be compared on the occurrence of specific answer behaviour (for instance giving ‘don’t know’-answers). By means of simulations, we show that the adapted profile-based Cliff’s Delta converges towards the original observation-based Cliff’s Delta as the number of items that is filled out by respondents increases. We conclude that the profile-based Cliff’s Delta is a solid and conservative statistic that is both useful and advantageous to compare behaviour profiles. We close with various data examples to illustrate its broad usefulness and by discussing potential difficulty in using the profile-based Cliff’s Delta.
|
| first_indexed | 2024-03-11T18:34:29Z |
| format | Article |
| id | doaj.art-f84084ac63b74a75a16508a17cd209ba |
| institution | Directory Open Access Journal |
| issn | 1864-3361 |
| language | English |
| last_indexed | 2024-03-11T18:34:29Z |
| publishDate | 2022-12-01 |
| publisher | European Survey Research Association |
| record_format | Article |
| series | Survey Research Methods |
| spelling | doaj.art-f84084ac63b74a75a16508a17cd209ba2023-10-13T07:35:21ZengEuropean Survey Research AssociationSurvey Research Methods1864-33612022-12-0116310.18148/srm/2022.v16i2.7908Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour ProfilesFrank Bais0Joost van der Neut1Cito Institute for Educational MeasurementDelft University of Technology Cliff’s Delta is a non-parametric effect size that is based on data observations. In this paper, we introduce an adaptation of Cliff’s Delta in order to compare behaviour profiles. Behaviour profiles are density distributions in which survey answer behaviour is summarized for specific groups of respondents or items. Such profiles are useful, as they take into account the varying number of survey items that is filled out per respondent due to filter questions. By the adapted Cliff’s Delta, two subgroups of respondents (for instance higher and lower educated respondents) can be compared on the occurrence of specific answer behaviour (for instance giving ‘don’t know’-answers). By means of simulations, we show that the adapted profile-based Cliff’s Delta converges towards the original observation-based Cliff’s Delta as the number of items that is filled out by respondents increases. We conclude that the profile-based Cliff’s Delta is a solid and conservative statistic that is both useful and advantageous to compare behaviour profiles. We close with various data examples to illustrate its broad usefulness and by discussing potential difficulty in using the profile-based Cliff’s Delta. https://ojs.ub.uni-konstanz.de/srm/article/view/7908Cliff's Deltabehaviour profilesundesirable answer behaviourmeasurement errorrespondent characteristicsitem characteristics |
| spellingShingle | Frank Bais Joost van der Neut Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles Survey Research Methods Cliff's Delta behaviour profiles undesirable answer behaviour measurement error respondent characteristics item characteristics |
| title | Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles |
| title_full | Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles |
| title_fullStr | Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles |
| title_full_unstemmed | Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles |
| title_short | Adapting the Robust Effect Size Cliff's Delta to Compare Behaviour Profiles |
| title_sort | adapting the robust effect size cliff s delta to compare behaviour profiles |
| topic | Cliff's Delta behaviour profiles undesirable answer behaviour measurement error respondent characteristics item characteristics |
| url | https://ojs.ub.uni-konstanz.de/srm/article/view/7908 |
| work_keys_str_mv | AT frankbais adaptingtherobusteffectsizecliffsdeltatocomparebehaviourprofiles AT joostvanderneut adaptingtherobusteffectsizecliffsdeltatocomparebehaviourprofiles |