Measurement practices in user experience (UX) research: a systematic quantitative literature review
User experience (UX) research relies heavily on survey scales to measure users' subjective experiences with technology. However, repeatedly raised concerns regarding the improper use of survey scales in UX research and adjacent fields call for a systematic review of current measurement practice...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-03-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1368860/full |
| _version_ | 1827330771100106752 |
|---|---|
| author | Sebastian A. C. Perrig Lena Fanya Aeschbach Nicolas Scharowski Nick von Felten Klaus Opwis Florian Brühlmann |
| author_facet | Sebastian A. C. Perrig Lena Fanya Aeschbach Nicolas Scharowski Nick von Felten Klaus Opwis Florian Brühlmann |
| author_sort | Sebastian A. C. Perrig |
| collection | DOAJ |
| description | User experience (UX) research relies heavily on survey scales to measure users' subjective experiences with technology. However, repeatedly raised concerns regarding the improper use of survey scales in UX research and adjacent fields call for a systematic review of current measurement practices. Therefore, we conducted a systematic literature review, screening 153 papers from four years of the ACM Conference on Human Factors in Computing Systems proceedings (ACM CHI 2019 to 2022), of which 60 were eligible empirical studies using survey scales to study users' experiences. We identified 85 different scales and 172 distinct constructs measured. Most scales were used once (70.59%), and most constructs were measured only once (66.28%). The System Usability Scale was the most popular scale, followed by the User Experience Questionnaire, and the NASA Task Load Index. Regarding constructs, usability was the most frequently measured, followed by attractiveness, effort, and presence. Furthermore, results show that papers rarely contained complete rationales for scale selection (20.00%) and seldom provided all scale items used (30.00%). More than a third of all scales were adapted (34.19%), while only one-third of papers reported any scale quality investigation (36.67%). On the basis of our results, we highlight questionable measurement practices in UX research and suggest opportunities to improve scale use for UX-related constructs. Additionally, we provide six recommended steps to promote enhanced rigor in following best practices for scale-based UX research. |
| first_indexed | 2024-03-07T16:22:13Z |
| format | Article |
| id | doaj.art-f0fc89d4b0ec4a8197af101c292a760c |
| institution | Directory Open Access Journal |
| issn | 2624-9898 |
| language | English |
| last_indexed | 2024-03-07T16:22:13Z |
| publishDate | 2024-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj.art-f0fc89d4b0ec4a8197af101c292a760c2024-03-04T04:52:55ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982024-03-01610.3389/fcomp.2024.13688601368860Measurement practices in user experience (UX) research: a systematic quantitative literature reviewSebastian A. C. PerrigLena Fanya AeschbachNicolas ScharowskiNick von FeltenKlaus OpwisFlorian BrühlmannUser experience (UX) research relies heavily on survey scales to measure users' subjective experiences with technology. However, repeatedly raised concerns regarding the improper use of survey scales in UX research and adjacent fields call for a systematic review of current measurement practices. Therefore, we conducted a systematic literature review, screening 153 papers from four years of the ACM Conference on Human Factors in Computing Systems proceedings (ACM CHI 2019 to 2022), of which 60 were eligible empirical studies using survey scales to study users' experiences. We identified 85 different scales and 172 distinct constructs measured. Most scales were used once (70.59%), and most constructs were measured only once (66.28%). The System Usability Scale was the most popular scale, followed by the User Experience Questionnaire, and the NASA Task Load Index. Regarding constructs, usability was the most frequently measured, followed by attractiveness, effort, and presence. Furthermore, results show that papers rarely contained complete rationales for scale selection (20.00%) and seldom provided all scale items used (30.00%). More than a third of all scales were adapted (34.19%), while only one-third of papers reported any scale quality investigation (36.67%). On the basis of our results, we highlight questionable measurement practices in UX research and suggest opportunities to improve scale use for UX-related constructs. Additionally, we provide six recommended steps to promote enhanced rigor in following best practices for scale-based UX research.https://www.frontiersin.org/articles/10.3389/fcomp.2024.1368860/fullliterature reviewquestionnairessurvey scalesuser experience (UX)measurementconstructs |
| spellingShingle | Sebastian A. C. Perrig Lena Fanya Aeschbach Nicolas Scharowski Nick von Felten Klaus Opwis Florian Brühlmann Measurement practices in user experience (UX) research: a systematic quantitative literature review Frontiers in Computer Science literature review questionnaires survey scales user experience (UX) measurement constructs |
| title | Measurement practices in user experience (UX) research: a systematic quantitative literature review |
| title_full | Measurement practices in user experience (UX) research: a systematic quantitative literature review |
| title_fullStr | Measurement practices in user experience (UX) research: a systematic quantitative literature review |
| title_full_unstemmed | Measurement practices in user experience (UX) research: a systematic quantitative literature review |
| title_short | Measurement practices in user experience (UX) research: a systematic quantitative literature review |
| title_sort | measurement practices in user experience ux research a systematic quantitative literature review |
| topic | literature review questionnaires survey scales user experience (UX) measurement constructs |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2024.1368860/full |
| work_keys_str_mv | AT sebastianacperrig measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview AT lenafanyaaeschbach measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview AT nicolasscharowski measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview AT nickvonfelten measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview AT klausopwis measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview AT florianbruhlmann measurementpracticesinuserexperienceuxresearchasystematicquantitativeliteraturereview |