Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis
Abstract Background The System Usability Scale (SUS) is used to measure usability of internet-based Cognitive Behavioural Therapy (iCBT). However, whether the SUS is a valid instrument to measure usability in this context is unclear. The aim of this study is to assess the factor structure of the SUS...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-05-01
|
| Series: | BMC Psychiatry |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12888-020-02627-8 |
| _version_ | 1818955751985512448 |
|---|---|
| author | Mayke Mol Anneke van Schaik Els Dozeman Jeroen Ruwaard Christiaan Vis David D. Ebert Anne Etzelmueller Kim Mathiasen Bárbara Moles Teresa Mora Claus D. Pedersen Mette Maria Skjøth Luisa Peleteiro Pensado Jordi Piera-Jimenez Didem Gokcay Burçin Ünlü Ince Alessio Russi Ylenia Sacco Enrico Zanalda Ane Fullaondo Zabala Heleen Riper Jan H. Smit |
| author_facet | Mayke Mol Anneke van Schaik Els Dozeman Jeroen Ruwaard Christiaan Vis David D. Ebert Anne Etzelmueller Kim Mathiasen Bárbara Moles Teresa Mora Claus D. Pedersen Mette Maria Skjøth Luisa Peleteiro Pensado Jordi Piera-Jimenez Didem Gokcay Burçin Ünlü Ince Alessio Russi Ylenia Sacco Enrico Zanalda Ane Fullaondo Zabala Heleen Riper Jan H. Smit |
| author_sort | Mayke Mol |
| collection | DOAJ |
| description | Abstract Background The System Usability Scale (SUS) is used to measure usability of internet-based Cognitive Behavioural Therapy (iCBT). However, whether the SUS is a valid instrument to measure usability in this context is unclear. The aim of this study is to assess the factor structure of the SUS, measuring usability of iCBT for depression in a sample of professionals. In addition, the psychometric properties (reliability, convergent validity) of the SUS were tested. Methods A sample of 242 professionals using iCBT for depression from 6 European countries completed the SUS. Confirmatory Factor Analysis (CFA) was conducted to test whether a one-factor, two-factor, tone-model or bi-direct model would fit the data best. Reliability was assessed using complementary statistical indices (e.g. omega). To assess convergent validity, the SUS total score was correlated with an adapted Client Satisfaction Questionnaire (CSQ-3). Results CFA supported the one-factor, two-factor and tone-model, but the bi-factor model fitted the data best (Comparative Fit Index = 0.992, Tucker Lewis Index = 0.985, Root Mean Square Error of Approximation = 0.055, Standardized Root Mean Square Residual = 0.042 (respectively χ2 diff (9) = 69.82, p < 0.001; χ2 diff (8) = 33.04, p < 0.001). Reliability of the SUS was good (ω = 0.91). The total SUS score correlated moderately with the CSQ-3 (CSQ1 rs = .49, p < 0.001; CSQ2 rs = .46, p < 0.001; CSQ3 rs = .38, p < 0.001), indicating convergent validity. Conclusions Although the SUS seems to have a multidimensional structure, the best model showed that the total sumscore of the SUS appears to be a valid and interpretable measure to assess the usability of internet-based interventions when used by professionals in mental healthcare. |
| first_indexed | 2024-12-20T10:43:03Z |
| format | Article |
| id | doaj.art-4301e2c1286043f99c3b3feb808aed38 |
| institution | Directory Open Access Journal |
| issn | 1471-244X |
| language | English |
| last_indexed | 2024-12-20T10:43:03Z |
| publishDate | 2020-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Psychiatry |
| spelling | doaj.art-4301e2c1286043f99c3b3feb808aed382022-12-21T19:43:31ZengBMCBMC Psychiatry1471-244X2020-05-0120111010.1186/s12888-020-02627-8Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysisMayke Mol0Anneke van Schaik1Els Dozeman2Jeroen Ruwaard3Christiaan Vis4David D. Ebert5Anne Etzelmueller6Kim Mathiasen7Bárbara Moles8Teresa Mora9Claus D. Pedersen10Mette Maria Skjøth11Luisa Peleteiro Pensado12Jordi Piera-Jimenez13Didem Gokcay14Burçin Ünlü Ince15Alessio Russi16Ylenia Sacco17Enrico Zanalda18Ane Fullaondo Zabala19Heleen Riper20Jan H. Smit21Department of Research and Innovation, GGZ inGeest, Specialized Mental Health CareDepartment of Research and Innovation, GGZ inGeest, Specialized Mental Health CareDepartment of Research and Innovation, GGZ inGeest, Specialized Mental Health CareDepartment of Research and Innovation, GGZ inGeest, Specialized Mental Health CareDepartment of Clinical, Neuro and Developmental Psychology, Clinical Psychology Section, Vrije Universiteit Amsterdam and the Amsterdam Public Health Research InstituteFriedrich-Alexander-Universität Erlangen-NürnbergFriedrich-Alexander-Universität Erlangen-NürnbergResearch Unit for Telepsychiatry and e-Mental Health, Department of Clinical Research, University of Southern DenmarkMental Health Unit, Barbastro Healthcare Sector, Aragón Healthcare Service –SALUDMental Health Unit, Barbastro Healthcare Sector, Aragón Healthcare Service –SALUDCentre for Innovative Medical Technology, Odense University HospitalCentre for Innovative Medical Technology, Odense University HospitalThe Public Health Service of Galicia -SERGAS, Service of Mental Health and Drug Addiction, Edificio Administrativo de San LázaroDepartment of Research and Innovation, Badalona Serveis AssistencialsDepartment Medical Informatics, Middle East Technical University, Informatics Institute, Üniversiteler MahallesiDepartment Medical Informatics, Middle East Technical University, Informatics Institute, Üniversiteler MahallesiULSS9 TrevisoMental Health Department, ASLTO3, Local Health Authority Torino 3Mental Health Department, ASLTO3, Local Health Authority Torino 3Institute of health service research - Kronikgunem, Torre del Bilbao Exhibition CentreDepartment of Research and Innovation, GGZ inGeest, Specialized Mental Health CareDepartment of Research and Innovation, GGZ inGeest, Specialized Mental Health CareAbstract Background The System Usability Scale (SUS) is used to measure usability of internet-based Cognitive Behavioural Therapy (iCBT). However, whether the SUS is a valid instrument to measure usability in this context is unclear. The aim of this study is to assess the factor structure of the SUS, measuring usability of iCBT for depression in a sample of professionals. In addition, the psychometric properties (reliability, convergent validity) of the SUS were tested. Methods A sample of 242 professionals using iCBT for depression from 6 European countries completed the SUS. Confirmatory Factor Analysis (CFA) was conducted to test whether a one-factor, two-factor, tone-model or bi-direct model would fit the data best. Reliability was assessed using complementary statistical indices (e.g. omega). To assess convergent validity, the SUS total score was correlated with an adapted Client Satisfaction Questionnaire (CSQ-3). Results CFA supported the one-factor, two-factor and tone-model, but the bi-factor model fitted the data best (Comparative Fit Index = 0.992, Tucker Lewis Index = 0.985, Root Mean Square Error of Approximation = 0.055, Standardized Root Mean Square Residual = 0.042 (respectively χ2 diff (9) = 69.82, p < 0.001; χ2 diff (8) = 33.04, p < 0.001). Reliability of the SUS was good (ω = 0.91). The total SUS score correlated moderately with the CSQ-3 (CSQ1 rs = .49, p < 0.001; CSQ2 rs = .46, p < 0.001; CSQ3 rs = .38, p < 0.001), indicating convergent validity. Conclusions Although the SUS seems to have a multidimensional structure, the best model showed that the total sumscore of the SUS appears to be a valid and interpretable measure to assess the usability of internet-based interventions when used by professionals in mental healthcare.http://link.springer.com/article/10.1186/s12888-020-02627-8Internet interventionsDepressionSystem usability scalePsychometric evaluationConfirmatory factor analysis |
| spellingShingle | Mayke Mol Anneke van Schaik Els Dozeman Jeroen Ruwaard Christiaan Vis David D. Ebert Anne Etzelmueller Kim Mathiasen Bárbara Moles Teresa Mora Claus D. Pedersen Mette Maria Skjøth Luisa Peleteiro Pensado Jordi Piera-Jimenez Didem Gokcay Burçin Ünlü Ince Alessio Russi Ylenia Sacco Enrico Zanalda Ane Fullaondo Zabala Heleen Riper Jan H. Smit Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis BMC Psychiatry Internet interventions Depression System usability scale Psychometric evaluation Confirmatory factor analysis |
| title | Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis |
| title_full | Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis |
| title_fullStr | Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis |
| title_full_unstemmed | Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis |
| title_short | Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis |
| title_sort | dimensionality of the system usability scale among professionals using internet based interventions for depression a confirmatory factor analysis |
| topic | Internet interventions Depression System usability scale Psychometric evaluation Confirmatory factor analysis |
| url | http://link.springer.com/article/10.1186/s12888-020-02627-8 |
| work_keys_str_mv | AT maykemol dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT annekevanschaik dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT elsdozeman dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT jeroenruwaard dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT christiaanvis dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT daviddebert dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT anneetzelmueller dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT kimmathiasen dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT barbaramoles dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT teresamora dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT clausdpedersen dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT mettemariaskjøth dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT luisapeleteiropensado dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT jordipierajimenez dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT didemgokcay dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT burcinunluince dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT alessiorussi dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT yleniasacco dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT enricozanalda dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT anefullaondozabala dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT heleenriper dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis AT janhsmit dimensionalityofthesystemusabilityscaleamongprofessionalsusinginternetbasedinterventionsfordepressionaconfirmatoryfactoranalysis |