Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans
COVID pandemic across the world and the emergence of new variants have intensified the need to identify COVID-19 cases quickly and efficiently. In this paper, a novel dual-mode multi-modal approach is presented to detect a covid patient. This has been done using the combination of image of the chest...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-02-01
|
| Series: | Intelligent Systems with Applications |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667305322000977 |
| _version_ | 1811171028281327616 |
|---|---|
| author | Nida Nasir Afreen Kansal Feras Barneih Omar Al-Shaltone Talal Bonny Mohammad Al-Shabi Ahmed Al Shammaa |
| author_facet | Nida Nasir Afreen Kansal Feras Barneih Omar Al-Shaltone Talal Bonny Mohammad Al-Shabi Ahmed Al Shammaa |
| author_sort | Nida Nasir |
| collection | DOAJ |
| description | COVID pandemic across the world and the emergence of new variants have intensified the need to identify COVID-19 cases quickly and efficiently. In this paper, a novel dual-mode multi-modal approach is presented to detect a covid patient. This has been done using the combination of image of the chest X-ray/CT scan and the clinical notes provided with the scan. Data augmentation techniques are used to extrapolate the dataset. Five different types of image and text models have been employed, including transfer learning. The binary cross entropy loss function and the adam optimizer are used to compile all of these models. The multi-modal is also tried out with existing pre-trained models such as: VGG16, ResNet50, InceptionResNetV2 and MobileNetV2. The final multi-modal gives an accuracy of 97.8% on the testing data. The study provides a different approach to identifying COVID-19 cases using just the scan images and the corresponding notes. |
| first_indexed | 2024-04-10T17:06:41Z |
| format | Article |
| id | doaj.art-cfc1fda2545e45e0ad3816123ef71db0 |
| institution | Directory Open Access Journal |
| issn | 2667-3053 |
| language | English |
| last_indexed | 2024-04-10T17:06:41Z |
| publishDate | 2023-02-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Intelligent Systems with Applications |
| spelling | doaj.art-cfc1fda2545e45e0ad3816123ef71db02023-02-06T04:06:22ZengElsevierIntelligent Systems with Applications2667-30532023-02-0117200160Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scansNida Nasir0Afreen Kansal1Feras Barneih2Omar Al-Shaltone3Talal Bonny4Mohammad Al-Shabi5Ahmed Al Shammaa6Research Institute of Science and Engineering, University of Sharjah, Sharjah, UAE; Corresponding author.Department of Statistics, London School of Economics and Political Science, London, UKResearch Institute of Science and Engineering, University of Sharjah, Sharjah, UAEResearch Institute of Science and Engineering, University of Sharjah, Sharjah, UAEResearch Institute of Science and Engineering, University of Sharjah, Sharjah, UAE; College of Computing and Informatics, University of Sharjah, Sharjah, UAEResearch Institute of Science and Engineering, University of Sharjah, Sharjah, UAE; College of Engineering, University of Sharjah, Sharjah, UAEResearch Institute of Science and Engineering, University of Sharjah, Sharjah, UAE; Department of Statistics, London School of Economics and Political Science, London, UK; College of Computing and Informatics, University of Sharjah, Sharjah, UAE; College of Engineering, University of Sharjah, Sharjah, UAE; Khorfakkan University, Khorfakkan, UAECOVID pandemic across the world and the emergence of new variants have intensified the need to identify COVID-19 cases quickly and efficiently. In this paper, a novel dual-mode multi-modal approach is presented to detect a covid patient. This has been done using the combination of image of the chest X-ray/CT scan and the clinical notes provided with the scan. Data augmentation techniques are used to extrapolate the dataset. Five different types of image and text models have been employed, including transfer learning. The binary cross entropy loss function and the adam optimizer are used to compile all of these models. The multi-modal is also tried out with existing pre-trained models such as: VGG16, ResNet50, InceptionResNetV2 and MobileNetV2. The final multi-modal gives an accuracy of 97.8% on the testing data. The study provides a different approach to identifying COVID-19 cases using just the scan images and the corresponding notes.http://www.sciencedirect.com/science/article/pii/S2667305322000977Machine learningTransfer learningAdam optimiserBinary cross entropy lossData augmentation |
| spellingShingle | Nida Nasir Afreen Kansal Feras Barneih Omar Al-Shaltone Talal Bonny Mohammad Al-Shabi Ahmed Al Shammaa Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans Intelligent Systems with Applications Machine learning Transfer learning Adam optimiser Binary cross entropy loss Data augmentation |
| title | Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans |
| title_full | Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans |
| title_fullStr | Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans |
| title_full_unstemmed | Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans |
| title_short | Multi-modal image classification of COVID-19 cases using computed tomography and X-rays scans |
| title_sort | multi modal image classification of covid 19 cases using computed tomography and x rays scans |
| topic | Machine learning Transfer learning Adam optimiser Binary cross entropy loss Data augmentation |
| url | http://www.sciencedirect.com/science/article/pii/S2667305322000977 |
| work_keys_str_mv | AT nidanasir multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT afreenkansal multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT ferasbarneih multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT omaralshaltone multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT talalbonny multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT mohammadalshabi multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans AT ahmedalshammaa multimodalimageclassificationofcovid19casesusingcomputedtomographyandxraysscans |