Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks
In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/22/6666 |
| _version_ | 1797547201625325568 |
|---|---|
| author | Kamil Książek Michał Romaszewski Przemysław Głomb Bartosz Grabowski Michał Cholewa |
| author_facet | Kamil Książek Michał Romaszewski Przemysław Głomb Bartosz Grabowski Michał Cholewa |
| author_sort | Kamil Książek |
| collection | DOAJ |
| description | In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem. |
| first_indexed | 2024-03-10T14:40:58Z |
| format | Article |
| id | doaj.art-6c80a8f5fd434951ae55bcf20d530ba7 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T14:40:58Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-6c80a8f5fd434951ae55bcf20d530ba72023-11-20T21:48:21ZengMDPI AGSensors1424-82202020-11-012022666610.3390/s20226666Blood Stain Classification with Hyperspectral Imaging and Deep Neural NetworksKamil Książek0Michał Romaszewski1Przemysław Głomb2Bartosz Grabowski3Michał Cholewa4Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, 44-100 Gliwice, PolandInstitute of Theoretical and Applied Informatics, Polish Academy of Sciences, 44-100 Gliwice, PolandInstitute of Theoretical and Applied Informatics, Polish Academy of Sciences, 44-100 Gliwice, PolandInstitute of Theoretical and Applied Informatics, Polish Academy of Sciences, 44-100 Gliwice, PolandInstitute of Theoretical and Applied Informatics, Polish Academy of Sciences, 44-100 Gliwice, PolandIn recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem.https://www.mdpi.com/1424-8220/20/22/6666hyperspectral classificationdeep learningdeep neural networksforensic scienceconvolutional neural networksrecurrent neural network |
| spellingShingle | Kamil Książek Michał Romaszewski Przemysław Głomb Bartosz Grabowski Michał Cholewa Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks Sensors hyperspectral classification deep learning deep neural networks forensic science convolutional neural networks recurrent neural network |
| title | Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks |
| title_full | Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks |
| title_fullStr | Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks |
| title_full_unstemmed | Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks |
| title_short | Blood Stain Classification with Hyperspectral Imaging and Deep Neural Networks |
| title_sort | blood stain classification with hyperspectral imaging and deep neural networks |
| topic | hyperspectral classification deep learning deep neural networks forensic science convolutional neural networks recurrent neural network |
| url | https://www.mdpi.com/1424-8220/20/22/6666 |
| work_keys_str_mv | AT kamilksiazek bloodstainclassificationwithhyperspectralimaginganddeepneuralnetworks AT michałromaszewski bloodstainclassificationwithhyperspectralimaginganddeepneuralnetworks AT przemysławgłomb bloodstainclassificationwithhyperspectralimaginganddeepneuralnetworks AT bartoszgrabowski bloodstainclassificationwithhyperspectralimaginganddeepneuralnetworks AT michałcholewa bloodstainclassificationwithhyperspectralimaginganddeepneuralnetworks |