A combination of two methods for evaluating the usability of a hospital information system
Abstract Background None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for com...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-05-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12911-020-1083-6 |
| _version_ | 1818150679087153152 |
|---|---|
| author | Reza Khajouei Fatemeh Farahani |
| author_facet | Reza Khajouei Fatemeh Farahani |
| author_sort | Reza Khajouei |
| collection | DOAJ |
| description | Abstract Background None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system. Methods Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods. Results The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the “Major” level. Conclusion Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems. |
| first_indexed | 2024-12-11T13:26:45Z |
| format | Article |
| id | doaj.art-77be462d549d4856a55507455bd80077 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-12-11T13:26:45Z |
| publishDate | 2020-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-77be462d549d4856a55507455bd800772022-12-22T01:05:29ZengBMCBMC Medical Informatics and Decision Making1472-69472020-05-0120111010.1186/s12911-020-1083-6A combination of two methods for evaluating the usability of a hospital information systemReza Khajouei0Fatemeh Farahani1Department of Health Information Sciences, Faculty of Management and Medical Information Sciences, Kerman University of Medical SciencesDepartment of Health Information Sciences, Faculty of Management and Medical Information Sciences, Kerman University of Medical SciencesAbstract Background None of the evaluation methods can identify all the usability problems of information systems. So far, no study has sufficiently investigated the potential of a combination of these methods to identify usability problems. The present study aimed at examining the potential for combining two commonly utilized user-based and expert-based methods to evaluate the usability of a hospital information system. Methods Think aloud (TA) and Heuristic evaluation (HE) methods were used to identify the usability problems of two subsystems of the Social Security Electronic System in Iran. To this end, the problems were categorized into five groups based on ISO-Nielsen usability attributes. The Chi-square test was applied to compare the intended methods based on the total number of problems and the number of problems within each group, followed by utilizing the Mann-Whitney U test to compare the mean severity scores of these methods. Results The evaluation by combining these methods yielded 423 problems of which 75% varied between the methods. The two methods were significantly different in terms of the total number of problems, the number of problems in each usability group, and the mean severity of two satisfaction and efficiency attributes (P < 0.05). However, no significant difference was observed between the two methods based on the mean severity of problems and severity scores related to three usability attributes i.e., effectiveness, learnability, and error prevention (P > 0.05). In addition, the mean severity of problems identified by each method was at the “Major” level. Conclusion Based on the results, although the mean severity scores of the identified problems were not significantly different, these methods identify heterogeneous problems. HE mainly identifies problems related to satisfaction, learnability, and error prevention while TA detects problems related to effectiveness and efficiency attributes. Therefore, using a combination of these two methods can identify a wider range of usability problems.http://link.springer.com/article/10.1186/s12911-020-1083-6Usability evaluationHuman-computer interactionHeuristic evaluationThink aloudHospital information system |
| spellingShingle | Reza Khajouei Fatemeh Farahani A combination of two methods for evaluating the usability of a hospital information system BMC Medical Informatics and Decision Making Usability evaluation Human-computer interaction Heuristic evaluation Think aloud Hospital information system |
| title | A combination of two methods for evaluating the usability of a hospital information system |
| title_full | A combination of two methods for evaluating the usability of a hospital information system |
| title_fullStr | A combination of two methods for evaluating the usability of a hospital information system |
| title_full_unstemmed | A combination of two methods for evaluating the usability of a hospital information system |
| title_short | A combination of two methods for evaluating the usability of a hospital information system |
| title_sort | a combination of two methods for evaluating the usability of a hospital information system |
| topic | Usability evaluation Human-computer interaction Heuristic evaluation Think aloud Hospital information system |
| url | http://link.springer.com/article/10.1186/s12911-020-1083-6 |
| work_keys_str_mv | AT rezakhajouei acombinationoftwomethodsforevaluatingtheusabilityofahospitalinformationsystem AT fatemehfarahani acombinationoftwomethodsforevaluatingtheusabilityofahospitalinformationsystem |