Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review
BackgroundIn the context of a deepening global shortage of health workers and, in particular, the COVID-19 pandemic, there is growing international interest in, and use of, online symptom checkers (OSCs). However, the evidence surrounding the triage and diagnostic accuracy of...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-06-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2023/1/e43803 |
| _version_ | 1827857296256925696 |
|---|---|
| author | Eva Riboli-Sasco Austen El-Osta Aos Alaa Iman Webber Manisha Karki Marie Line El Asmar Katie Purohit Annabelle Painter Benedict Hayhoe |
| author_facet | Eva Riboli-Sasco Austen El-Osta Aos Alaa Iman Webber Manisha Karki Marie Line El Asmar Katie Purohit Annabelle Painter Benedict Hayhoe |
| author_sort | Eva Riboli-Sasco |
| collection | DOAJ |
| description |
BackgroundIn the context of a deepening global shortage of health workers and, in particular, the COVID-19 pandemic, there is growing international interest in, and use of, online symptom checkers (OSCs). However, the evidence surrounding the triage and diagnostic accuracy of these tools remains inconclusive.
ObjectiveThis systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns.
MethodsSearches were conducted in MEDLINE, Embase, CINAHL, Health Management Information Consortium (HMIC), and Web of Science, as well as the citations of the studies selected for full-text screening. We included peer-reviewed studies published in English between January 1, 2010, and February 16, 2022, with a controlled and quantitative assessment of either or both triage and diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health care professionals, as well as disease- or specialty-specific OSCs. Screening and data extraction were carried out independently by 2 reviewers for each study. We performed a descriptive narrative synthesis.
ResultsA total of 21,296 studies were identified, of which 14 (0.07%) were included. The included studies used clinical vignettes, medical records, or direct input by patients. Of the 14 studies, 6 (43%) reported on triage and diagnostic accuracy, 7 (50%) focused on triage accuracy, and 1 (7%) focused on diagnostic accuracy. These outcomes were assessed based on the diagnostic and triage recommendations attached to the vignette in the case of vignette studies or on those provided by nurses or general practitioners, including through face-to-face and telephone consultations. Both diagnostic accuracy and triage accuracy varied greatly among OSCs. Overall diagnostic accuracy was deemed to be low and was almost always lower than that of the comparator. Similarly, most of the studies (9/13, 69 %) showed suboptimal triage accuracy overall, with a few exceptions (4/13, 31%). The main variables affecting the levels of diagnostic and triage accuracy were the severity and urgency of the condition, the use of artificial intelligence algorithms, and demographic questions. However, the impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. All included studies had at least one area with unclear risk of bias according to the revised Quality Assessment of Diagnostic Accuracy Studies-2 tool.
ConclusionsAlthough OSCs have potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy before widescale adoption in community and health care settings. Future studies should aim to use a common methodology and agreed standard for evaluation to facilitate objective benchmarking and validation.
Trial RegistrationPROSPERO CRD42020215210; https://tinyurl.com/3949zw83 |
| first_indexed | 2024-03-12T12:38:27Z |
| format | Article |
| id | doaj.art-7e58a246aaa74176bf6d470664802a9e |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T12:38:27Z |
| publishDate | 2023-06-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-7e58a246aaa74176bf6d470664802a9e2023-08-29T00:00:29ZengJMIR PublicationsJournal of Medical Internet Research1438-88712023-06-0125e4380310.2196/43803Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic ReviewEva Riboli-Sascohttps://orcid.org/0000-0002-9922-9125Austen El-Ostahttps://orcid.org/0000-0002-8772-4938Aos Alaahttps://orcid.org/0000-0002-4757-2562Iman Webberhttps://orcid.org/0000-0002-1690-0121Manisha Karkihttps://orcid.org/0000-0001-6996-8915Marie Line El Asmarhttps://orcid.org/0000-0002-8481-6431Katie Purohithttps://orcid.org/0000-0001-6518-3627Annabelle Painterhttps://orcid.org/0000-0003-2383-0181Benedict Hayhoehttps://orcid.org/0000-0002-2645-6191 BackgroundIn the context of a deepening global shortage of health workers and, in particular, the COVID-19 pandemic, there is growing international interest in, and use of, online symptom checkers (OSCs). However, the evidence surrounding the triage and diagnostic accuracy of these tools remains inconclusive. ObjectiveThis systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns. MethodsSearches were conducted in MEDLINE, Embase, CINAHL, Health Management Information Consortium (HMIC), and Web of Science, as well as the citations of the studies selected for full-text screening. We included peer-reviewed studies published in English between January 1, 2010, and February 16, 2022, with a controlled and quantitative assessment of either or both triage and diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health care professionals, as well as disease- or specialty-specific OSCs. Screening and data extraction were carried out independently by 2 reviewers for each study. We performed a descriptive narrative synthesis. ResultsA total of 21,296 studies were identified, of which 14 (0.07%) were included. The included studies used clinical vignettes, medical records, or direct input by patients. Of the 14 studies, 6 (43%) reported on triage and diagnostic accuracy, 7 (50%) focused on triage accuracy, and 1 (7%) focused on diagnostic accuracy. These outcomes were assessed based on the diagnostic and triage recommendations attached to the vignette in the case of vignette studies or on those provided by nurses or general practitioners, including through face-to-face and telephone consultations. Both diagnostic accuracy and triage accuracy varied greatly among OSCs. Overall diagnostic accuracy was deemed to be low and was almost always lower than that of the comparator. Similarly, most of the studies (9/13, 69 %) showed suboptimal triage accuracy overall, with a few exceptions (4/13, 31%). The main variables affecting the levels of diagnostic and triage accuracy were the severity and urgency of the condition, the use of artificial intelligence algorithms, and demographic questions. However, the impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. All included studies had at least one area with unclear risk of bias according to the revised Quality Assessment of Diagnostic Accuracy Studies-2 tool. ConclusionsAlthough OSCs have potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy before widescale adoption in community and health care settings. Future studies should aim to use a common methodology and agreed standard for evaluation to facilitate objective benchmarking and validation. Trial RegistrationPROSPERO CRD42020215210; https://tinyurl.com/3949zw83https://www.jmir.org/2023/1/e43803 |
| spellingShingle | Eva Riboli-Sasco Austen El-Osta Aos Alaa Iman Webber Manisha Karki Marie Line El Asmar Katie Purohit Annabelle Painter Benedict Hayhoe Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review Journal of Medical Internet Research |
| title | Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review |
| title_full | Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review |
| title_fullStr | Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review |
| title_full_unstemmed | Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review |
| title_short | Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review |
| title_sort | triage and diagnostic accuracy of online symptom checkers systematic review |
| url | https://www.jmir.org/2023/1/e43803 |
| work_keys_str_mv | AT evaribolisasco triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT austenelosta triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT aosalaa triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT imanwebber triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT manishakarki triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT marielineelasmar triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT katiepurohit triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT annabellepainter triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview AT benedicthayhoe triageanddiagnosticaccuracyofonlinesymptomcheckerssystematicreview |