Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models
The modern scientific world continuously endeavors to battle and devise solutions for newly arising pandemics. One such pandemic which has turned the world’s accustomed routine upside down is COVID-19: it has devastated the world economy and destroyed around 45 million lives, globally. Governments a...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/13/6364 |
| _version_ | 1827655573504524288 |
|---|---|
| author | Ali Tariq Nagi Mazhar Javed Awan Mazin Abed Mohammed Amena Mahmoud Arnab Majumdar Orawit Thinnukool |
| author_facet | Ali Tariq Nagi Mazhar Javed Awan Mazin Abed Mohammed Amena Mahmoud Arnab Majumdar Orawit Thinnukool |
| author_sort | Ali Tariq Nagi |
| collection | DOAJ |
| description | The modern scientific world continuously endeavors to battle and devise solutions for newly arising pandemics. One such pandemic which has turned the world’s accustomed routine upside down is COVID-19: it has devastated the world economy and destroyed around 45 million lives, globally. Governments and scientists have been on the front line, striving towards the diagnosis and engineering of a vaccination for the said virus. COVID-19 can be diagnosed using artificial intelligence more accurately than traditional methods using chest X-rays. This research involves an evaluation of the performance of deep learning models for COVID-19 diagnosis using chest X-ray images from a dataset containing the largest number of COVID-19 images ever used in the literature, according to the best of the authors’ knowledge. The size of the utilized dataset is about 4.25 times the maximum COVID-19 chest X-ray image dataset used in the explored literature. Further, a CNN model was developed, named the Custom-Model in this study, for evaluation against, and comparison to, the state-of-the-art deep learning models. The intention was not to develop a new high-performing deep learning model, but rather to evaluate the performance of deep learning models on a larger COVID-19 chest X-ray image dataset. Moreover, Xception- and MobilNetV2- based models were also used for evaluation purposes. The criteria for evaluation were based on accuracy, precision, recall, F1 score, ROC curves, AUC, confusion matrix, and macro and weighted averages. Among the deployed models, Xception was the top performer in terms of precision and accuracy, while the MobileNetV2-based model could detect slightly more COVID-19 cases than Xception, and showed slightly fewer false negatives, while giving far more false positives than the other models. Also, the custom CNN model exceeds the MobileNetV2 model in terms of precision. The best accuracy, precision, recall, and F1 score out of these three models were 94.2%, 99%, 95%, and 97%, respectively, as shown by the Xception model. Finally, it was found that the overall accuracy in the current evaluation was curtailed by approximately 2% compared with the average accuracy of previous work on multi-class classification, while a very high precision value was observed, which is of high scientific value. |
| first_indexed | 2024-03-09T22:09:34Z |
| format | Article |
| id | doaj.art-97af68b611194d14b9d6e9c69af6acf2 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T22:09:34Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-97af68b611194d14b9d6e9c69af6acf22023-11-23T19:35:21ZengMDPI AGApplied Sciences2076-34172022-06-011213636410.3390/app12136364Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning ModelsAli Tariq Nagi0Mazhar Javed Awan1Mazin Abed Mohammed2Amena Mahmoud3Arnab Majumdar4Orawit Thinnukool5Department of Software Engineering, University of Management and Technology, Lahore 54770, PakistanDepartment of Software Engineering, University of Management and Technology, Lahore 54770, PakistanCollege of Computer Science and Information Technology, University of Anbar, Anbar 31001, IraqComputer Science Department, Faculty of Computers and Information, Kafrelsheikh University, Kafr Al-Sheikh 33516, EgyptFaculty of Engineering, Imperial College London, London SW7 2AZ, UKCollege of Arts, Media and Technology, Chiang Mai University, Chiang Mai 50200, ThailandThe modern scientific world continuously endeavors to battle and devise solutions for newly arising pandemics. One such pandemic which has turned the world’s accustomed routine upside down is COVID-19: it has devastated the world economy and destroyed around 45 million lives, globally. Governments and scientists have been on the front line, striving towards the diagnosis and engineering of a vaccination for the said virus. COVID-19 can be diagnosed using artificial intelligence more accurately than traditional methods using chest X-rays. This research involves an evaluation of the performance of deep learning models for COVID-19 diagnosis using chest X-ray images from a dataset containing the largest number of COVID-19 images ever used in the literature, according to the best of the authors’ knowledge. The size of the utilized dataset is about 4.25 times the maximum COVID-19 chest X-ray image dataset used in the explored literature. Further, a CNN model was developed, named the Custom-Model in this study, for evaluation against, and comparison to, the state-of-the-art deep learning models. The intention was not to develop a new high-performing deep learning model, but rather to evaluate the performance of deep learning models on a larger COVID-19 chest X-ray image dataset. Moreover, Xception- and MobilNetV2- based models were also used for evaluation purposes. The criteria for evaluation were based on accuracy, precision, recall, F1 score, ROC curves, AUC, confusion matrix, and macro and weighted averages. Among the deployed models, Xception was the top performer in terms of precision and accuracy, while the MobileNetV2-based model could detect slightly more COVID-19 cases than Xception, and showed slightly fewer false negatives, while giving far more false positives than the other models. Also, the custom CNN model exceeds the MobileNetV2 model in terms of precision. The best accuracy, precision, recall, and F1 score out of these three models were 94.2%, 99%, 95%, and 97%, respectively, as shown by the Xception model. Finally, it was found that the overall accuracy in the current evaluation was curtailed by approximately 2% compared with the average accuracy of previous work on multi-class classification, while a very high precision value was observed, which is of high scientific value.https://www.mdpi.com/2076-3417/12/13/6364artificial Intelligencechest X-rayCOVID-19deep learningconvolutional neural networkslung opacity |
| spellingShingle | Ali Tariq Nagi Mazhar Javed Awan Mazin Abed Mohammed Amena Mahmoud Arnab Majumdar Orawit Thinnukool Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models Applied Sciences artificial Intelligence chest X-ray COVID-19 deep learning convolutional neural networks lung opacity |
| title | Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models |
| title_full | Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models |
| title_fullStr | Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models |
| title_full_unstemmed | Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models |
| title_short | Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models |
| title_sort | performance analysis for covid 19 diagnosis using custom and state of the art deep learning models |
| topic | artificial Intelligence chest X-ray COVID-19 deep learning convolutional neural networks lung opacity |
| url | https://www.mdpi.com/2076-3417/12/13/6364 |
| work_keys_str_mv | AT alitariqnagi performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels AT mazharjavedawan performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels AT mazinabedmohammed performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels AT amenamahmoud performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels AT arnabmajumdar performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels AT orawitthinnukool performanceanalysisforcovid19diagnosisusingcustomandstateoftheartdeeplearningmodels |