Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists
Abstract Background Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-05-01
|
| Series: | BioMedical Engineering OnLine |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12938-022-01001-x |
| _version_ | 1818206974412587008 |
|---|---|
| author | Adrian Krenzer Kevin Makowski Amar Hekalo Daniel Fitting Joel Troya Wolfram G. Zoller Alexander Hann Frank Puppe |
| author_facet | Adrian Krenzer Kevin Makowski Amar Hekalo Daniel Fitting Joel Troya Wolfram G. Zoller Alexander Hann Frank Puppe |
| author_sort | Adrian Krenzer |
| collection | DOAJ |
| description | Abstract Background Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between. Methods In our framework, an expert reviews the video and annotates a few video frames to verify the object’s annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model. Results Using this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool. Conclusion In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source. |
| first_indexed | 2024-12-12T04:21:33Z |
| format | Article |
| id | doaj.art-9bb60f9ba4e24448b4c5e16b850475d9 |
| institution | Directory Open Access Journal |
| issn | 1475-925X |
| language | English |
| last_indexed | 2024-12-12T04:21:33Z |
| publishDate | 2022-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BioMedical Engineering OnLine |
| spelling | doaj.art-9bb60f9ba4e24448b4c5e16b850475d92022-12-22T00:38:18ZengBMCBioMedical Engineering OnLine1475-925X2022-05-0121112310.1186/s12938-022-01001-xFast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologistsAdrian Krenzer0Kevin Makowski1Amar Hekalo2Daniel Fitting3Joel Troya4Wolfram G. Zoller5Alexander Hann6Frank Puppe7Department of Artificial Intelligence and Knowledge SystemsDepartment of Artificial Intelligence and Knowledge SystemsDepartment of Artificial Intelligence and Knowledge SystemsInterventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital WürzburgInterventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital WürzburgDepartment of Internal Medicine and Gastroenterology, KatharinenhospitalInterventional and Experimental Endoscopy (InExEn), Department of Internal Medicine II, University Hospital WürzburgDepartment of Artificial Intelligence and Knowledge SystemsAbstract Background Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between. Methods In our framework, an expert reviews the video and annotates a few video frames to verify the object’s annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model. Results Using this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool. Conclusion In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source.https://doi.org/10.1186/s12938-022-01001-xMachine learningDeep learningAnnotationEndoscopyGastroenterologyAutomation |
| spellingShingle | Adrian Krenzer Kevin Makowski Amar Hekalo Daniel Fitting Joel Troya Wolfram G. Zoller Alexander Hann Frank Puppe Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists BioMedical Engineering OnLine Machine learning Deep learning Annotation Endoscopy Gastroenterology Automation |
| title | Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists |
| title_full | Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists |
| title_fullStr | Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists |
| title_full_unstemmed | Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists |
| title_short | Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists |
| title_sort | fast machine learning annotation in the medical domain a semi automated video annotation tool for gastroenterologists |
| topic | Machine learning Deep learning Annotation Endoscopy Gastroenterology Automation |
| url | https://doi.org/10.1186/s12938-022-01001-x |
| work_keys_str_mv | AT adriankrenzer fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT kevinmakowski fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT amarhekalo fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT danielfitting fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT joeltroya fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT wolframgzoller fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT alexanderhann fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists AT frankpuppe fastmachinelearningannotationinthemedicaldomainasemiautomatedvideoannotationtoolforgastroenterologists |