Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures
Forced-choice (FC) measures are gaining popularity as an alternative assessment format to single-statement (SS) measures. However, a fundamental question remains to be answered: Do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cogniti...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/150191 |
| _version_ | 1826129609012281344 |
|---|---|
| author | Zhang, Bo Sun, Tianjun Drasgow, Fritz Chernyshenko, Oleksandr S. Nye, Christopher D. Stark, Stephen White, Leonard A. |
| author2 | Nanyang Business School |
| author_facet | Nanyang Business School Zhang, Bo Sun, Tianjun Drasgow, Fritz Chernyshenko, Oleksandr S. Nye, Christopher D. Stark, Stephen White, Leonard A. |
| author_sort | Zhang, Bo |
| collection | NTU |
| description | Forced-choice (FC) measures are gaining popularity as an alternative assessment format to single-statement (SS) measures. However, a fundamental question remains to be answered: Do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cognitively challenging, so how would this feature influence respondents’ reactions to FC measures compared to SS? We used both between- and within-subjects designs to examine the equivalence of the FC format and the SS format. As the results illustrate, FC measures scored by the multi-unidimensional pairwise preference (MUPP) model and SS measures scored with the generalized graded unfolding model (GGUM) showed strong equivalence. Specifically, both formats demonstrated similar marginal reliabilities and test-retest reliabilities, high convergent validities, good discriminant validities, and similar criterion-related validities with theoretically relevant criteria. In addition, the formats had little differential impact on respondents’ general emotional and cognitive reactions except that the FC format was perceived to be slightly more difficult and more time-saving. |
| first_indexed | 2024-10-01T07:43:20Z |
| format | Journal Article |
| id | ntu-10356/150191 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:43:20Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | ntu-10356/1501912023-05-19T07:31:16Z Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures Zhang, Bo Sun, Tianjun Drasgow, Fritz Chernyshenko, Oleksandr S. Nye, Christopher D. Stark, Stephen White, Leonard A. Nanyang Business School Social sciences::Psychology Forced-choice Single-statement Forced-choice (FC) measures are gaining popularity as an alternative assessment format to single-statement (SS) measures. However, a fundamental question remains to be answered: Do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cognitively challenging, so how would this feature influence respondents’ reactions to FC measures compared to SS? We used both between- and within-subjects designs to examine the equivalence of the FC format and the SS format. As the results illustrate, FC measures scored by the multi-unidimensional pairwise preference (MUPP) model and SS measures scored with the generalized graded unfolding model (GGUM) showed strong equivalence. Specifically, both formats demonstrated similar marginal reliabilities and test-retest reliabilities, high convergent validities, good discriminant validities, and similar criterion-related validities with theoretically relevant criteria. In addition, the formats had little differential impact on respondents’ general emotional and cognitive reactions except that the FC format was perceived to be slightly more difficult and more time-saving. 2021-06-09T04:04:13Z 2021-06-09T04:04:13Z 2020 Journal Article Zhang, B., Sun, T., Drasgow, F., Chernyshenko, O. S., Nye, C. D., Stark, S. & White, L. A. (2020). Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures. Organizational Research Methods, 23(3), 569-590. https://dx.doi.org/10.1177/1094428119836486 1094-4281 0000-0002-6730-7336 https://hdl.handle.net/10356/150191 10.1177/1094428119836486 2-s2.0-85064047034 3 23 569 590 en Organizational Research Methods © 2019 The Author(s). All rights reserved. |
| spellingShingle | Social sciences::Psychology Forced-choice Single-statement Zhang, Bo Sun, Tianjun Drasgow, Fritz Chernyshenko, Oleksandr S. Nye, Christopher D. Stark, Stephen White, Leonard A. Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title | Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title_full | Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title_fullStr | Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title_full_unstemmed | Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title_short | Though forced, still valid : psychometric equivalence of forced-choice and single-statement measures |
| title_sort | though forced still valid psychometric equivalence of forced choice and single statement measures |
| topic | Social sciences::Psychology Forced-choice Single-statement |
| url | https://hdl.handle.net/10356/150191 |
| work_keys_str_mv | AT zhangbo thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT suntianjun thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT drasgowfritz thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT chernyshenkooleksandrs thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT nyechristopherd thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT starkstephen thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures AT whiteleonarda thoughforcedstillvalidpsychometricequivalenceofforcedchoiceandsinglestatementmeasures |