The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study
BackgroundThe recognition and interpretation of abnormal blood cell morphology is often the first step in diagnosing underlying serious systemic illness or leukemia. Supporting the staff who interpret blood film morphology is therefore essential for a safe laboratory service....
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-08-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2021/8/e20815 |
| _version_ | 1797735812785242112 |
|---|---|
| author | Claire Hutchinson Michelle Brereton Julie Adams Barbara De La Salle Jon Sims Keith Hyde Richard Chasty Rachel Brown Karen Rees-Unwin John Burthem |
| author_facet | Claire Hutchinson Michelle Brereton Julie Adams Barbara De La Salle Jon Sims Keith Hyde Richard Chasty Rachel Brown Karen Rees-Unwin John Burthem |
| author_sort | Claire Hutchinson |
| collection | DOAJ |
| description |
BackgroundThe recognition and interpretation of abnormal blood cell morphology is often the first step in diagnosing underlying serious systemic illness or leukemia. Supporting the staff who interpret blood film morphology is therefore essential for a safe laboratory service. This paper describes an open-access, web-based decision support tool, developed by the authors to support morphological diagnosis, arising from earlier studies identifying mechanisms of error in blood film reporting. The effectiveness of this intervention was assessed using the unique resource offered by the online digital morphology Continuing Professional Development scheme (DM scheme) offered by the UK National External Quality Assessment Service for Haematology, with more than 3000 registered users. This allowed the effectiveness of decision support to be tested within a defined user group, each of whom viewed and interpreted the morphology of identical digital blood films.
ObjectiveThe primary objective of the study was to test the effectiveness of the decision support system in supporting users to identify and interpret abnormal morphological features. The secondary objective was to determine the pattern and frequency of use of the system for different case types, and to determine how users perceived the support in terms of their confidence in decision-making.
MethodsThis was a comparative study of identical blood films evaluated either with or without decision support. Selected earlier cases from the DM scheme were rereleased as new cases but with decision support made available; this allowed a comparison of data sets for identical cases with or without decision support. To address the primary objectives, the study used quantitative evaluation and statistical comparisons of the identification and interpretation of morphological features between the two different case releases. To address the secondary objective, the use of decision support was assessed using web analytical tools, while a questionnaire was used to assess user perceptions of the system.
ResultsCases evaluated with the aid of decision support had significantly improved accuracy of identification for relevant morphological features (mean improvement 9.8%) and the interpretation of those features (mean improvement 11%). The improvement was particularly significant for cases with higher complexity or for rarer diagnoses. Analysis of website usage demonstrated a high frequency of access for web pages relevant to each case (mean 9298 for each case, range 2661-24,276). Users reported that the decision support website increased their confidence for feature identification (4.8/5) and interpretation (4.3/5), both within the context of training (4.6/5) and also in their wider laboratory practice (4.4/5).
ConclusionsThe findings of this study demonstrate that directed online decision support for blood morphology evaluation improves accuracy and confidence in the context of educational evaluation of digital films, with effectiveness potentially extending to wider laboratory use. |
| first_indexed | 2024-03-12T13:04:31Z |
| format | Article |
| id | doaj.art-8910409152c44dbfa145c4fbb2708a4e |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T13:04:31Z |
| publishDate | 2021-08-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-8910409152c44dbfa145c4fbb2708a4e2023-08-28T18:28:58ZengJMIR PublicationsJournal of Medical Internet Research1438-88712021-08-01238e2081510.2196/20815The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational StudyClaire Hutchinsonhttps://orcid.org/0000-0003-2314-6558Michelle Breretonhttps://orcid.org/0000-0003-0584-909XJulie Adamshttps://orcid.org/0000-0001-5953-5622Barbara De La Sallehttps://orcid.org/0000-0002-2926-7441Jon Simshttps://orcid.org/0000-0002-3172-5173Keith Hydehttps://orcid.org/0000-0003-2137-5955Richard Chastyhttps://orcid.org/0000-0001-5388-4565Rachel Brownhttps://orcid.org/0000-0001-8956-8724Karen Rees-Unwinhttps://orcid.org/0000-0002-3088-2695John Burthemhttps://orcid.org/0000-0002-7361-9328 BackgroundThe recognition and interpretation of abnormal blood cell morphology is often the first step in diagnosing underlying serious systemic illness or leukemia. Supporting the staff who interpret blood film morphology is therefore essential for a safe laboratory service. This paper describes an open-access, web-based decision support tool, developed by the authors to support morphological diagnosis, arising from earlier studies identifying mechanisms of error in blood film reporting. The effectiveness of this intervention was assessed using the unique resource offered by the online digital morphology Continuing Professional Development scheme (DM scheme) offered by the UK National External Quality Assessment Service for Haematology, with more than 3000 registered users. This allowed the effectiveness of decision support to be tested within a defined user group, each of whom viewed and interpreted the morphology of identical digital blood films. ObjectiveThe primary objective of the study was to test the effectiveness of the decision support system in supporting users to identify and interpret abnormal morphological features. The secondary objective was to determine the pattern and frequency of use of the system for different case types, and to determine how users perceived the support in terms of their confidence in decision-making. MethodsThis was a comparative study of identical blood films evaluated either with or without decision support. Selected earlier cases from the DM scheme were rereleased as new cases but with decision support made available; this allowed a comparison of data sets for identical cases with or without decision support. To address the primary objectives, the study used quantitative evaluation and statistical comparisons of the identification and interpretation of morphological features between the two different case releases. To address the secondary objective, the use of decision support was assessed using web analytical tools, while a questionnaire was used to assess user perceptions of the system. ResultsCases evaluated with the aid of decision support had significantly improved accuracy of identification for relevant morphological features (mean improvement 9.8%) and the interpretation of those features (mean improvement 11%). The improvement was particularly significant for cases with higher complexity or for rarer diagnoses. Analysis of website usage demonstrated a high frequency of access for web pages relevant to each case (mean 9298 for each case, range 2661-24,276). Users reported that the decision support website increased their confidence for feature identification (4.8/5) and interpretation (4.3/5), both within the context of training (4.6/5) and also in their wider laboratory practice (4.4/5). ConclusionsThe findings of this study demonstrate that directed online decision support for blood morphology evaluation improves accuracy and confidence in the context of educational evaluation of digital films, with effectiveness potentially extending to wider laboratory use.https://www.jmir.org/2021/8/e20815 |
| spellingShingle | Claire Hutchinson Michelle Brereton Julie Adams Barbara De La Salle Jon Sims Keith Hyde Richard Chasty Rachel Brown Karen Rees-Unwin John Burthem The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study Journal of Medical Internet Research |
| title | The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study |
| title_full | The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study |
| title_fullStr | The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study |
| title_full_unstemmed | The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study |
| title_short | The Use and Effectiveness of an Online Diagnostic Support System for Blood Film Interpretation: Comparative Observational Study |
| title_sort | use and effectiveness of an online diagnostic support system for blood film interpretation comparative observational study |
| url | https://www.jmir.org/2021/8/e20815 |
| work_keys_str_mv | AT clairehutchinson theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT michellebrereton theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT julieadams theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT barbaradelasalle theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT jonsims theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT keithhyde theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT richardchasty theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT rachelbrown theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT karenreesunwin theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT johnburthem theuseandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT clairehutchinson useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT michellebrereton useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT julieadams useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT barbaradelasalle useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT jonsims useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT keithhyde useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT richardchasty useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT rachelbrown useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT karenreesunwin useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy AT johnburthem useandeffectivenessofanonlinediagnosticsupportsystemforbloodfilminterpretationcomparativeobservationalstudy |