Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification
Deep Learning is a very active and important area for building Computer-Aided Diagnosis (CAD) applications. This work aims to present a hybrid model to classify lung ultrasound (LUS) videos captured by convex transducers to diagnose COVID-19. A Convolutional Neural Network (CNN) performed the extrac...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/16/5486 |
| _version_ | 1797522068543111168 |
|---|---|
| author | Bruno Barros Paulo Lacerda Célio Albuquerque Aura Conci |
| author_facet | Bruno Barros Paulo Lacerda Célio Albuquerque Aura Conci |
| author_sort | Bruno Barros |
| collection | DOAJ |
| description | Deep Learning is a very active and important area for building Computer-Aided Diagnosis (CAD) applications. This work aims to present a hybrid model to classify lung ultrasound (LUS) videos captured by convex transducers to diagnose COVID-19. A Convolutional Neural Network (CNN) performed the extraction of spatial features, and the temporal dependence was learned using a Long Short-Term Memory (LSTM). Different types of convolutional architectures were used for feature extraction. The hybrid model (CNN-LSTM) hyperparameters were optimized using the Optuna framework. The best hybrid model was composed of an Xception pre-trained on ImageNet and an LSTM containing 512 units, configured with a dropout rate of 0.4, two fully connected layers containing 1024 neurons each, and a sequence of 20 frames in the input layer <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>(</mo><mn>20</mn><mo>×</mo><mn>2018</mn><mo>)</mo></mrow></semantics></math></inline-formula>. The model presented an average accuracy of 93% and sensitivity of 97% for COVID-19, outperforming models based purely on spatial approaches. Furthermore, feature extraction using transfer learning with models pre-trained on ImageNet provided comparable results to models pre-trained on LUS images. The results corroborate with other studies showing that this model for LUS classification can be an important tool in the fight against COVID-19 and other lung diseases. |
| first_indexed | 2024-03-10T08:24:19Z |
| format | Article |
| id | doaj.art-4cc540db8452421ab1bb78220d9f12a5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T08:24:19Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-4cc540db8452421ab1bb78220d9f12a52023-11-22T09:40:38ZengMDPI AGSensors1424-82202021-08-012116548610.3390/s21165486Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video ClassificationBruno Barros0Paulo Lacerda1Célio Albuquerque2Aura Conci3Institute of Computing, Campus Praia Vermelha, Fluminense Federal University, Niterói 24.210-346, BrazilInstitute of Computing, Campus Praia Vermelha, Fluminense Federal University, Niterói 24.210-346, BrazilInstitute of Computing, Campus Praia Vermelha, Fluminense Federal University, Niterói 24.210-346, BrazilInstitute of Computing, Campus Praia Vermelha, Fluminense Federal University, Niterói 24.210-346, BrazilDeep Learning is a very active and important area for building Computer-Aided Diagnosis (CAD) applications. This work aims to present a hybrid model to classify lung ultrasound (LUS) videos captured by convex transducers to diagnose COVID-19. A Convolutional Neural Network (CNN) performed the extraction of spatial features, and the temporal dependence was learned using a Long Short-Term Memory (LSTM). Different types of convolutional architectures were used for feature extraction. The hybrid model (CNN-LSTM) hyperparameters were optimized using the Optuna framework. The best hybrid model was composed of an Xception pre-trained on ImageNet and an LSTM containing 512 units, configured with a dropout rate of 0.4, two fully connected layers containing 1024 neurons each, and a sequence of 20 frames in the input layer <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>(</mo><mn>20</mn><mo>×</mo><mn>2018</mn><mo>)</mo></mrow></semantics></math></inline-formula>. The model presented an average accuracy of 93% and sensitivity of 97% for COVID-19, outperforming models based purely on spatial approaches. Furthermore, feature extraction using transfer learning with models pre-trained on ImageNet provided comparable results to models pre-trained on LUS images. The results corroborate with other studies showing that this model for LUS classification can be an important tool in the fight against COVID-19 and other lung diseases.https://www.mdpi.com/1424-8220/21/16/5486COVID-19CNNDeep LearningLSTMlung ultrasoundneural networks |
| spellingShingle | Bruno Barros Paulo Lacerda Célio Albuquerque Aura Conci Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification Sensors COVID-19 CNN Deep Learning LSTM lung ultrasound neural networks |
| title | Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification |
| title_full | Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification |
| title_fullStr | Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification |
| title_full_unstemmed | Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification |
| title_short | Pulmonary COVID-19: Learning Spatiotemporal Features Combining CNN and LSTM Networks for Lung Ultrasound Video Classification |
| title_sort | pulmonary covid 19 learning spatiotemporal features combining cnn and lstm networks for lung ultrasound video classification |
| topic | COVID-19 CNN Deep Learning LSTM lung ultrasound neural networks |
| url | https://www.mdpi.com/1424-8220/21/16/5486 |
| work_keys_str_mv | AT brunobarros pulmonarycovid19learningspatiotemporalfeaturescombiningcnnandlstmnetworksforlungultrasoundvideoclassification AT paulolacerda pulmonarycovid19learningspatiotemporalfeaturescombiningcnnandlstmnetworksforlungultrasoundvideoclassification AT celioalbuquerque pulmonarycovid19learningspatiotemporalfeaturescombiningcnnandlstmnetworksforlungultrasoundvideoclassification AT auraconci pulmonarycovid19learningspatiotemporalfeaturescombiningcnnandlstmnetworksforlungultrasoundvideoclassification |