Explainable Multimedia Feature Fusion for Medical Applications
Due to the exponential growth of medical information in the form of, e.g., text, images, Electrocardiograms (ECGs), X-rays, and multimedia, the management of a patient’s data has become a huge challenge. In particular, the extraction of features from various different formats and their representatio...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/8/4/104 |
| _version_ | 1827619662622359552 |
|---|---|
| author | Stefan Wagenpfeil Paul Mc Kevitt Abbas Cheddad Matthias Hemmje |
| author_facet | Stefan Wagenpfeil Paul Mc Kevitt Abbas Cheddad Matthias Hemmje |
| author_sort | Stefan Wagenpfeil |
| collection | DOAJ |
| description | Due to the exponential growth of medical information in the form of, e.g., text, images, Electrocardiograms (ECGs), X-rays, and multimedia, the management of a patient’s data has become a huge challenge. In particular, the extraction of features from various different formats and their representation in a homogeneous way are areas of interest in medical applications. Multimedia Information Retrieval (MMIR) frameworks, like the Generic Multimedia Analysis Framework (GMAF), can contribute to solving this problem, when adapted to special requirements and modalities of medical applications. In this paper, we demonstrate how typical multimedia processing techniques can be extended and adapted to medical applications and how these applications benefit from employing a Multimedia Feature Graph (MMFG) and specialized, efficient indexing structures in the form of Graph Codes. These Graph Codes are transformed to feature relevant Graph Codes by employing a modified Term Frequency Inverse Document Frequency (TFIDF) algorithm, which further supports value ranges and Boolean operations required in the medical context. On this basis, various metrics for the calculation of similarity, recommendations, and automated inferencing and reasoning can be applied supporting the field of diagnostics. Finally, the presentation of these new facilities in the form of explainability is introduced and demonstrated. Thus, in this paper, we show how Graph Codes contribute new querying options for diagnosis and how Explainable Graph Codes can help to readily understand medical multimedia formats. |
| first_indexed | 2024-03-09T10:33:17Z |
| format | Article |
| id | doaj.art-9b1c74aa9d9745aba6b029dd64d790f6 |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-09T10:33:17Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-9b1c74aa9d9745aba6b029dd64d790f62023-12-01T21:07:38ZengMDPI AGJournal of Imaging2313-433X2022-04-018410410.3390/jimaging8040104Explainable Multimedia Feature Fusion for Medical ApplicationsStefan Wagenpfeil0Paul Mc Kevitt1Abbas Cheddad2Matthias Hemmje3Faculty of Mathematics and Computer Science, University of Hagen, Universitätsstrasse 1, 58097 Hagen, GermanyAcademy for International Science & Research (AISR), Derry BT48 7JL, UKBlekinge Institute of Technology, 371 79 Karlskrona, SwedenFaculty of Mathematics and Computer Science, University of Hagen, Universitätsstrasse 1, 58097 Hagen, GermanyDue to the exponential growth of medical information in the form of, e.g., text, images, Electrocardiograms (ECGs), X-rays, and multimedia, the management of a patient’s data has become a huge challenge. In particular, the extraction of features from various different formats and their representation in a homogeneous way are areas of interest in medical applications. Multimedia Information Retrieval (MMIR) frameworks, like the Generic Multimedia Analysis Framework (GMAF), can contribute to solving this problem, when adapted to special requirements and modalities of medical applications. In this paper, we demonstrate how typical multimedia processing techniques can be extended and adapted to medical applications and how these applications benefit from employing a Multimedia Feature Graph (MMFG) and specialized, efficient indexing structures in the form of Graph Codes. These Graph Codes are transformed to feature relevant Graph Codes by employing a modified Term Frequency Inverse Document Frequency (TFIDF) algorithm, which further supports value ranges and Boolean operations required in the medical context. On this basis, various metrics for the calculation of similarity, recommendations, and automated inferencing and reasoning can be applied supporting the field of diagnostics. Finally, the presentation of these new facilities in the form of explainability is introduced and demonstrated. Thus, in this paper, we show how Graph Codes contribute new querying options for diagnosis and how Explainable Graph Codes can help to readily understand medical multimedia formats.https://www.mdpi.com/2313-433X/8/4/104indexingretrievalexplainabilitysemanticmultimediafeature graph |
| spellingShingle | Stefan Wagenpfeil Paul Mc Kevitt Abbas Cheddad Matthias Hemmje Explainable Multimedia Feature Fusion for Medical Applications Journal of Imaging indexing retrieval explainability semantic multimedia feature graph |
| title | Explainable Multimedia Feature Fusion for Medical Applications |
| title_full | Explainable Multimedia Feature Fusion for Medical Applications |
| title_fullStr | Explainable Multimedia Feature Fusion for Medical Applications |
| title_full_unstemmed | Explainable Multimedia Feature Fusion for Medical Applications |
| title_short | Explainable Multimedia Feature Fusion for Medical Applications |
| title_sort | explainable multimedia feature fusion for medical applications |
| topic | indexing retrieval explainability semantic multimedia feature graph |
| url | https://www.mdpi.com/2313-433X/8/4/104 |
| work_keys_str_mv | AT stefanwagenpfeil explainablemultimediafeaturefusionformedicalapplications AT paulmckevitt explainablemultimediafeaturefusionformedicalapplications AT abbascheddad explainablemultimediafeaturefusionformedicalapplications AT matthiashemmje explainablemultimediafeaturefusionformedicalapplications |