Quantity over quality: Do single-item scales reflect what users truly experienced?
Single-item scales are widely used in the field of user experience to report the emotions. However, they are strongly criticized and discouraged by the scientific community. While they have several practical advantages, single-item scales are mainly criticized for their psychometric weakness. Our re...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2021-08-01
|
| Series: | Computers in Human Behavior Reports |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2451958821000452 |
| _version_ | 1798025186495168512 |
|---|---|
| author | Matthieu Cuvillier Pierre-Majorique Léger Sylvain Sénécal |
| author_facet | Matthieu Cuvillier Pierre-Majorique Léger Sylvain Sénécal |
| author_sort | Matthieu Cuvillier |
| collection | DOAJ |
| description | Single-item scales are widely used in the field of user experience to report the emotions. However, they are strongly criticized and discouraged by the scientific community. While they have several practical advantages, single-item scales are mainly criticized for their psychometric weakness. Our research explores to what extent single-item scales reflect what a user has experienced during his or her interaction with technology, overall, but also at specific moments such as first and last impressions. This research also explores the sensitivity of these measures across different contexts of use brought by the presence or absence of certain interface features. We conducted a correlational study with 40 users while interacting with financial institution websites. We used two methods to evaluate the experience: lived and measured implicitly using psychophysiological instruments on one hand and self-perceived and measured using single-item scales, on the other hand. Overall, our results suggest limited correlations and in most cases contradictory between lived and perceived experience. We conclude by highlighting the limits to be taken into account when professionals are led to use single-item scales. |
| first_indexed | 2024-04-11T18:14:41Z |
| format | Article |
| id | doaj.art-df9bd57307734c82afc57237e00b8170 |
| institution | Directory Open Access Journal |
| issn | 2451-9588 |
| language | English |
| last_indexed | 2024-04-11T18:14:41Z |
| publishDate | 2021-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Computers in Human Behavior Reports |
| spelling | doaj.art-df9bd57307734c82afc57237e00b81702022-12-22T04:09:58ZengElsevierComputers in Human Behavior Reports2451-95882021-08-014100097Quantity over quality: Do single-item scales reflect what users truly experienced?Matthieu Cuvillier0Pierre-Majorique Léger1Sylvain Sénécal2Corresponding author.; HEC Montréal, 3000 Chemin de la Côte-Sainte-Catherine, Montréal, QC, H3T 2A7, CanadaHEC Montréal, 3000 Chemin de la Côte-Sainte-Catherine, Montréal, QC, H3T 2A7, CanadaHEC Montréal, 3000 Chemin de la Côte-Sainte-Catherine, Montréal, QC, H3T 2A7, CanadaSingle-item scales are widely used in the field of user experience to report the emotions. However, they are strongly criticized and discouraged by the scientific community. While they have several practical advantages, single-item scales are mainly criticized for their psychometric weakness. Our research explores to what extent single-item scales reflect what a user has experienced during his or her interaction with technology, overall, but also at specific moments such as first and last impressions. This research also explores the sensitivity of these measures across different contexts of use brought by the presence or absence of certain interface features. We conducted a correlational study with 40 users while interacting with financial institution websites. We used two methods to evaluate the experience: lived and measured implicitly using psychophysiological instruments on one hand and self-perceived and measured using single-item scales, on the other hand. Overall, our results suggest limited correlations and in most cases contradictory between lived and perceived experience. We conclude by highlighting the limits to be taken into account when professionals are led to use single-item scales.http://www.sciencedirect.com/science/article/pii/S2451958821000452ArousalCognitive loadNet promoter scoreSatisfactionSingle-item scalesValence |
| spellingShingle | Matthieu Cuvillier Pierre-Majorique Léger Sylvain Sénécal Quantity over quality: Do single-item scales reflect what users truly experienced? Computers in Human Behavior Reports Arousal Cognitive load Net promoter score Satisfaction Single-item scales Valence |
| title | Quantity over quality: Do single-item scales reflect what users truly experienced? |
| title_full | Quantity over quality: Do single-item scales reflect what users truly experienced? |
| title_fullStr | Quantity over quality: Do single-item scales reflect what users truly experienced? |
| title_full_unstemmed | Quantity over quality: Do single-item scales reflect what users truly experienced? |
| title_short | Quantity over quality: Do single-item scales reflect what users truly experienced? |
| title_sort | quantity over quality do single item scales reflect what users truly experienced |
| topic | Arousal Cognitive load Net promoter score Satisfaction Single-item scales Valence |
| url | http://www.sciencedirect.com/science/article/pii/S2451958821000452 |
| work_keys_str_mv | AT matthieucuvillier quantityoverqualitydosingleitemscalesreflectwhatuserstrulyexperienced AT pierremajoriqueleger quantityoverqualitydosingleitemscalesreflectwhatuserstrulyexperienced AT sylvainsenecal quantityoverqualitydosingleitemscalesreflectwhatuserstrulyexperienced |