Evaluation of a clinical decision support system for rare diseases: a qualitative study
Abstract Background Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German univers...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2021-02-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-021-01435-8 |
| _version_ | 1828959903780175872 |
|---|---|
| author | Jannik Schaaf Martin Sedlmayr Brita Sedlmayr Hans-Ulrich Prokosch Holger Storf |
| author_facet | Jannik Schaaf Martin Sedlmayr Brita Sedlmayr Hans-Ulrich Prokosch Holger Storf |
| author_sort | Jannik Schaaf |
| collection | DOAJ |
| description | Abstract Background Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS. Methods We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS). Results A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability. Conclusions This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis. |
| first_indexed | 2024-12-14T09:16:16Z |
| format | Article |
| id | doaj.art-b7891171a1d84863b2980b78597a0ee5 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-12-14T09:16:16Z |
| publishDate | 2021-02-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-b7891171a1d84863b2980b78597a0ee52022-12-21T23:08:26ZengBMCBMC Medical Informatics and Decision Making1472-69472021-02-0121111110.1186/s12911-021-01435-8Evaluation of a clinical decision support system for rare diseases: a qualitative studyJannik Schaaf0Martin Sedlmayr1Brita Sedlmayr2Hans-Ulrich Prokosch3Holger Storf4Medical Informatics Group (MIG), University Hospital FrankfurtInstitute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technical University of DresdenInstitute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technical University of DresdenDepartment of Medical Informatics, Biometrics and Epidemiology, Friedrich-Alexander University Erlangen-NürnbergMedical Informatics Group (MIG), University Hospital FrankfurtAbstract Background Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS. Methods We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS). Results A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability. Conclusions This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.https://doi.org/10.1186/s12911-021-01435-8Rare diseasesClinical decision support systemsComputer-assisted diagnosisUsability |
| spellingShingle | Jannik Schaaf Martin Sedlmayr Brita Sedlmayr Hans-Ulrich Prokosch Holger Storf Evaluation of a clinical decision support system for rare diseases: a qualitative study BMC Medical Informatics and Decision Making Rare diseases Clinical decision support systems Computer-assisted diagnosis Usability |
| title | Evaluation of a clinical decision support system for rare diseases: a qualitative study |
| title_full | Evaluation of a clinical decision support system for rare diseases: a qualitative study |
| title_fullStr | Evaluation of a clinical decision support system for rare diseases: a qualitative study |
| title_full_unstemmed | Evaluation of a clinical decision support system for rare diseases: a qualitative study |
| title_short | Evaluation of a clinical decision support system for rare diseases: a qualitative study |
| title_sort | evaluation of a clinical decision support system for rare diseases a qualitative study |
| topic | Rare diseases Clinical decision support systems Computer-assisted diagnosis Usability |
| url | https://doi.org/10.1186/s12911-021-01435-8 |
| work_keys_str_mv | AT jannikschaaf evaluationofaclinicaldecisionsupportsystemforrarediseasesaqualitativestudy AT martinsedlmayr evaluationofaclinicaldecisionsupportsystemforrarediseasesaqualitativestudy AT britasedlmayr evaluationofaclinicaldecisionsupportsystemforrarediseasesaqualitativestudy AT hansulrichprokosch evaluationofaclinicaldecisionsupportsystemforrarediseasesaqualitativestudy AT holgerstorf evaluationofaclinicaldecisionsupportsystemforrarediseasesaqualitativestudy |