An investigation of gender-based differences in assessment instruments: A test of measurement invariance
Orientation: Practitioners and researchers often assume that the psychometric instruments they use are invariant and that they therefore measure similar constructs in a comparable manner across men and women respondents. This assumption is, however, rarely tested, leading to an undetected bias in re...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AOSIS
2020-03-01
|
| Series: | SA Journal of Industrial Psychology |
| Subjects: | |
| Online Access: | https://sajip.co.za/index.php/sajip/article/view/1699 |
| _version_ | 1818159507852754944 |
|---|---|
| author | Renier Steyn Gideon P. de Bruin |
| author_facet | Renier Steyn Gideon P. de Bruin |
| author_sort | Renier Steyn |
| collection | DOAJ |
| description | Orientation: Practitioners and researchers often assume that the psychometric instruments they use are invariant and that they therefore measure similar constructs in a comparable manner across men and women respondents. This assumption is, however, rarely tested, leading to an undetected bias in research findings or an adverse impact because of the presence of non-invariance.
Research purpose: After presenting essential information about measurement invariance (MI) and arguing for the testing thereof, this research aims to reveal the prevalence of MI across several frequently used psychometric instruments credulously used based on the assumption the revenant constructs are measured equivalently across gender exists.
Motivation for the study: Firstly, this study aims to increase awareness regarding MI, a property that can be tested statistically. Secondly, the research aims to make practitioners aware of the presence of bias in psychometric instruments, specifically to identify instruments that could be included in investigations which attempt to understand gender matters in the workplace.
Research approach/design and method: Cross-sectional survey data, pertaining to seven standard instruments, related to innovative work behaviour, were analysed. Pairwise, multigroup confirmatory factor analyses with robust maximum likelihood estimation were used to examine configural, metric, intercept and strict invariance, as well as the equivalence of the latent means.
Main findings: The findings were binary, with four of the instruments showing MI at an equal latent means level, whilst three instruments were non-invariant at the configural level. Measurement invariance was either accepted completely or rejected completely.
Practical/managerial implications: The serratedness of findings, even when using well-recognised and frequently used psychometric instruments, exposes the prevalence of non-invariance in some instruments, thereby necessitating the standard testing for MI. These findings also specify the instruments that are MI (in terms of gender), which allow other researchers and practitioners to use these instruments with more confidence when measuring and comparing men and women respondents in their studies.
Contribution/value-add: This research demonstrates the ease with which MI testing can be performed and alerts researchers to do MI testing when conducting cross-group studies, as the prevalence of measurement non-invariance is high. |
| first_indexed | 2024-12-11T15:47:05Z |
| format | Article |
| id | doaj.art-9b5812ba3ffe4e8ba0f6dd1c4a0d5e84 |
| institution | Directory Open Access Journal |
| issn | 0258-5200 2071-0763 |
| language | English |
| last_indexed | 2024-12-11T15:47:05Z |
| publishDate | 2020-03-01 |
| publisher | AOSIS |
| record_format | Article |
| series | SA Journal of Industrial Psychology |
| spelling | doaj.art-9b5812ba3ffe4e8ba0f6dd1c4a0d5e842022-12-22T00:59:40ZengAOSISSA Journal of Industrial Psychology0258-52002071-07632020-03-01460e1e1210.4102/sajip.v46i0.16991143An investigation of gender-based differences in assessment instruments: A test of measurement invarianceRenier Steyn0Gideon P. de Bruin1Department of Industrial Psychology and People Management, University of Johannesburg, JohannesburgDepartment of Industrial Psychology and People Management, University of Johannesburg, JohannesburgOrientation: Practitioners and researchers often assume that the psychometric instruments they use are invariant and that they therefore measure similar constructs in a comparable manner across men and women respondents. This assumption is, however, rarely tested, leading to an undetected bias in research findings or an adverse impact because of the presence of non-invariance. Research purpose: After presenting essential information about measurement invariance (MI) and arguing for the testing thereof, this research aims to reveal the prevalence of MI across several frequently used psychometric instruments credulously used based on the assumption the revenant constructs are measured equivalently across gender exists. Motivation for the study: Firstly, this study aims to increase awareness regarding MI, a property that can be tested statistically. Secondly, the research aims to make practitioners aware of the presence of bias in psychometric instruments, specifically to identify instruments that could be included in investigations which attempt to understand gender matters in the workplace. Research approach/design and method: Cross-sectional survey data, pertaining to seven standard instruments, related to innovative work behaviour, were analysed. Pairwise, multigroup confirmatory factor analyses with robust maximum likelihood estimation were used to examine configural, metric, intercept and strict invariance, as well as the equivalence of the latent means. Main findings: The findings were binary, with four of the instruments showing MI at an equal latent means level, whilst three instruments were non-invariant at the configural level. Measurement invariance was either accepted completely or rejected completely. Practical/managerial implications: The serratedness of findings, even when using well-recognised and frequently used psychometric instruments, exposes the prevalence of non-invariance in some instruments, thereby necessitating the standard testing for MI. These findings also specify the instruments that are MI (in terms of gender), which allow other researchers and practitioners to use these instruments with more confidence when measuring and comparing men and women respondents in their studies. Contribution/value-add: This research demonstrates the ease with which MI testing can be performed and alerts researchers to do MI testing when conducting cross-group studies, as the prevalence of measurement non-invariance is high.https://sajip.co.za/index.php/sajip/article/view/1699gendermeasurement invariancebiasadverse effectgroup differencesinnovative work behaviour. |
| spellingShingle | Renier Steyn Gideon P. de Bruin An investigation of gender-based differences in assessment instruments: A test of measurement invariance SA Journal of Industrial Psychology gender measurement invariance bias adverse effect group differences innovative work behaviour. |
| title | An investigation of gender-based differences in assessment instruments: A test of measurement invariance |
| title_full | An investigation of gender-based differences in assessment instruments: A test of measurement invariance |
| title_fullStr | An investigation of gender-based differences in assessment instruments: A test of measurement invariance |
| title_full_unstemmed | An investigation of gender-based differences in assessment instruments: A test of measurement invariance |
| title_short | An investigation of gender-based differences in assessment instruments: A test of measurement invariance |
| title_sort | investigation of gender based differences in assessment instruments a test of measurement invariance |
| topic | gender measurement invariance bias adverse effect group differences innovative work behaviour. |
| url | https://sajip.co.za/index.php/sajip/article/view/1699 |
| work_keys_str_mv | AT reniersteyn aninvestigationofgenderbaseddifferencesinassessmentinstrumentsatestofmeasurementinvariance AT gideonpdebruin aninvestigationofgenderbaseddifferencesinassessmentinstrumentsatestofmeasurementinvariance AT reniersteyn investigationofgenderbaseddifferencesinassessmentinstrumentsatestofmeasurementinvariance AT gideonpdebruin investigationofgenderbaseddifferencesinassessmentinstrumentsatestofmeasurementinvariance |