Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study
BackgroundPrehospitalization documentation is a challenging task and prone to loss of information, as paramedics operate under disruptive environments requiring their constant attention to the patients. ObjectiveThe aim of this study is to develop a mobile platfor...
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-10-01
|
| Series: | JMIR mHealth and uHealth |
| Online Access: | https://mhealth.jmir.org/2021/10/e32301 |
| _version_ | 1797735680253624320 |
|---|---|
| author | MinJae Woo Prabodh Mishra Ju Lin Snigdhaswin Kar Nicholas Deas Caleb Linduff Sufeng Niu Yuzhe Yang Jerome McClendon D Hudson Smith Stephen L Shelton Christopher E Gainey William C Gerard Melissa C Smith Sarah F Griffin Ronald W Gimbel Kuang-Ching Wang |
| author_facet | MinJae Woo Prabodh Mishra Ju Lin Snigdhaswin Kar Nicholas Deas Caleb Linduff Sufeng Niu Yuzhe Yang Jerome McClendon D Hudson Smith Stephen L Shelton Christopher E Gainey William C Gerard Melissa C Smith Sarah F Griffin Ronald W Gimbel Kuang-Ching Wang |
| author_sort | MinJae Woo |
| collection | DOAJ |
| description |
BackgroundPrehospitalization documentation is a challenging task and prone to loss of information, as paramedics operate under disruptive environments requiring their constant attention to the patients.
ObjectiveThe aim of this study is to develop a mobile platform for hands-free prehospitalization documentation to assist first responders in operational medical environments by aggregating all existing solutions for noise resiliency and domain adaptation.
MethodsThe platform was built to extract meaningful medical information from the real-time audio streaming at the point of injury and transmit complete documentation to a field hospital prior to patient arrival. To this end, the state-of-the-art automatic speech recognition (ASR) solutions with the following modular improvements were thoroughly explored: noise-resilient ASR, multi-style training, customized lexicon, and speech enhancement. The development of the platform was strictly guided by qualitative research and simulation-based evaluation to address the relevant challenges through progressive improvements at every process step of the end-to-end solution. The primary performance metrics included medical word error rate (WER) in machine-transcribed text output and an F1 score calculated by comparing the autogenerated documentation to manual documentation by physicians.
ResultsThe total number of 15,139 individual words necessary for completing the documentation were identified from all conversations that occurred during the physician-supervised simulation drills. The baseline model presented a suboptimal performance with a WER of 69.85% and an F1 score of 0.611. The noise-resilient ASR, multi-style training, and customized lexicon improved the overall performance; the finalized platform achieved a medical WER of 33.3% and an F1 score of 0.81 when compared to manual documentation. The speech enhancement degraded performance with medical WER increased from 33.3% to 46.33% and the corresponding F1 score decreased from 0.81 to 0.78. All changes in performance were statistically significant (P<.001).
ConclusionsThis study presented a fully functional mobile platform for hands-free prehospitalization documentation in operational medical environments and lessons learned from its implementation. |
| first_indexed | 2024-03-12T13:02:27Z |
| format | Article |
| id | doaj.art-91c6c09a9bee459c8bd9bde0583cdbf9 |
| institution | Directory Open Access Journal |
| issn | 2291-5222 |
| language | English |
| last_indexed | 2024-03-12T13:02:27Z |
| publishDate | 2021-10-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR mHealth and uHealth |
| spelling | doaj.art-91c6c09a9bee459c8bd9bde0583cdbf92023-08-28T19:30:47ZengJMIR PublicationsJMIR mHealth and uHealth2291-52222021-10-01910e3230110.2196/32301Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental StudyMinJae Woohttps://orcid.org/0000-0001-6015-2100Prabodh Mishrahttps://orcid.org/0000-0001-6944-9330Ju Linhttps://orcid.org/0000-0002-6970-4247Snigdhaswin Karhttps://orcid.org/0000-0003-4561-7998Nicholas Deashttps://orcid.org/0000-0002-2306-5101Caleb Linduffhttps://orcid.org/0000-0001-8554-6958Sufeng Niuhttps://orcid.org/0000-0002-2826-0301Yuzhe Yanghttps://orcid.org/0000-0001-9210-6002Jerome McClendonhttps://orcid.org/0000-0002-5435-0247D Hudson Smithhttps://orcid.org/0000-0003-3041-4602Stephen L Sheltonhttps://orcid.org/0000-0002-0350-5245Christopher E Gaineyhttps://orcid.org/0000-0003-3976-5644William C Gerardhttps://orcid.org/0000-0002-8111-7352Melissa C Smithhttps://orcid.org/0000-0003-0798-8536Sarah F Griffinhttps://orcid.org/0000-0003-4820-3985Ronald W Gimbelhttps://orcid.org/0000-0001-8185-4013Kuang-Ching Wanghttps://orcid.org/0000-0002-5675-7104 BackgroundPrehospitalization documentation is a challenging task and prone to loss of information, as paramedics operate under disruptive environments requiring their constant attention to the patients. ObjectiveThe aim of this study is to develop a mobile platform for hands-free prehospitalization documentation to assist first responders in operational medical environments by aggregating all existing solutions for noise resiliency and domain adaptation. MethodsThe platform was built to extract meaningful medical information from the real-time audio streaming at the point of injury and transmit complete documentation to a field hospital prior to patient arrival. To this end, the state-of-the-art automatic speech recognition (ASR) solutions with the following modular improvements were thoroughly explored: noise-resilient ASR, multi-style training, customized lexicon, and speech enhancement. The development of the platform was strictly guided by qualitative research and simulation-based evaluation to address the relevant challenges through progressive improvements at every process step of the end-to-end solution. The primary performance metrics included medical word error rate (WER) in machine-transcribed text output and an F1 score calculated by comparing the autogenerated documentation to manual documentation by physicians. ResultsThe total number of 15,139 individual words necessary for completing the documentation were identified from all conversations that occurred during the physician-supervised simulation drills. The baseline model presented a suboptimal performance with a WER of 69.85% and an F1 score of 0.611. The noise-resilient ASR, multi-style training, and customized lexicon improved the overall performance; the finalized platform achieved a medical WER of 33.3% and an F1 score of 0.81 when compared to manual documentation. The speech enhancement degraded performance with medical WER increased from 33.3% to 46.33% and the corresponding F1 score decreased from 0.81 to 0.78. All changes in performance were statistically significant (P<.001). ConclusionsThis study presented a fully functional mobile platform for hands-free prehospitalization documentation in operational medical environments and lessons learned from its implementation.https://mhealth.jmir.org/2021/10/e32301 |
| spellingShingle | MinJae Woo Prabodh Mishra Ju Lin Snigdhaswin Kar Nicholas Deas Caleb Linduff Sufeng Niu Yuzhe Yang Jerome McClendon D Hudson Smith Stephen L Shelton Christopher E Gainey William C Gerard Melissa C Smith Sarah F Griffin Ronald W Gimbel Kuang-Ching Wang Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study JMIR mHealth and uHealth |
| title | Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study |
| title_full | Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study |
| title_fullStr | Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study |
| title_full_unstemmed | Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study |
| title_short | Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study |
| title_sort | complete and resilient documentation for operational medical environments leveraging mobile hands free technology in a systems approach experimental study |
| url | https://mhealth.jmir.org/2021/10/e32301 |
| work_keys_str_mv | AT minjaewoo completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT prabodhmishra completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT julin completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT snigdhaswinkar completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT nicholasdeas completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT caleblinduff completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT sufengniu completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT yuzheyang completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT jeromemcclendon completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT dhudsonsmith completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT stephenlshelton completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT christopheregainey completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT williamcgerard completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT melissacsmith completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT sarahfgriffin completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT ronaldwgimbel completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy AT kuangchingwang completeandresilientdocumentationforoperationalmedicalenvironmentsleveragingmobilehandsfreetechnologyinasystemsapproachexperimentalstudy |