Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model
Abstract Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general pote...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-06-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-15040-w |
| _version_ | 1828335331650306048 |
|---|---|
| author | Alexander Studier-Fischer Silvia Seidlitz Jan Sellner Berkin Özdemir Manuel Wiesenfarth Leonardo Ayala Jan Odenthal Samuel Knödler Karl Friedrich Kowalewski Caelan Max Haney Isabella Camplisson Maximilian Dietrich Karsten Schmidt Gabriel Alexander Salg Hannes Götz Kenngott Tim Julian Adler Nicholas Schreck Annette Kopp-Schneider Klaus Maier-Hein Lena Maier-Hein Beat Peter Müller-Stich Felix Nickel |
| author_facet | Alexander Studier-Fischer Silvia Seidlitz Jan Sellner Berkin Özdemir Manuel Wiesenfarth Leonardo Ayala Jan Odenthal Samuel Knödler Karl Friedrich Kowalewski Caelan Max Haney Isabella Camplisson Maximilian Dietrich Karsten Schmidt Gabriel Alexander Salg Hannes Götz Kenngott Tim Julian Adler Nicholas Schreck Annette Kopp-Schneider Klaus Maier-Hein Lena Maier-Hein Beat Peter Müller-Stich Felix Nickel |
| author_sort | Alexander Studier-Fischer |
| collection | DOAJ |
| description | Abstract Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics. |
| first_indexed | 2024-04-13T21:44:58Z |
| format | Article |
| id | doaj.art-ebb9fc0545f64ebba5e2b0ae635ad0f5 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T21:44:58Z |
| publishDate | 2022-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-ebb9fc0545f64ebba5e2b0ae635ad0f52022-12-22T02:28:38ZengNature PortfolioScientific Reports2045-23222022-06-0112111410.1038/s41598-022-15040-wSpectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine modelAlexander Studier-Fischer0Silvia Seidlitz1Jan Sellner2Berkin Özdemir3Manuel Wiesenfarth4Leonardo Ayala5Jan Odenthal6Samuel Knödler7Karl Friedrich Kowalewski8Caelan Max Haney9Isabella Camplisson10Maximilian Dietrich11Karsten Schmidt12Gabriel Alexander Salg13Hannes Götz Kenngott14Tim Julian Adler15Nicholas Schreck16Annette Kopp-Schneider17Klaus Maier-Hein18Lena Maier-Hein19Beat Peter Müller-Stich20Felix Nickel21Department of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDivision of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Department of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDivision of Biostatistics, German Cancer Research Center (DKFZ)Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Department of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDepartment of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDepartment of Urology, Medical Faculty of Mannheim, University of HeidelbergDepartment of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDivision of Biology and Biological Engineering, California Institute of TechnologyDepartment of Anesthesiology, Heidelberg University HospitalDepartment of Anesthesiology and Intensive Care Medicine, Essen University HospitalDepartment of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDepartment of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDivision of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Division of Biostatistics, German Cancer Research Center (DKFZ)Division of Biostatistics, German Cancer Research Center (DKFZ)Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)Department of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalDepartment of General, Visceral, and Transplantation Surgery, Heidelberg University HospitalAbstract Visual discrimination of tissue during surgery can be challenging since different tissues appear similar to the human eye. Hyperspectral imaging (HSI) removes this limitation by associating each pixel with high-dimensional spectral information. While previous work has shown its general potential to discriminate tissue, clinical translation has been limited due to the method’s current lack of robustness and generalizability. Specifically, the scientific community is lacking a comprehensive spectral tissue atlas, and it is unknown whether variability in spectral reflectance is primarily explained by tissue type rather than the recorded individual or specific acquisition conditions. The contribution of this work is threefold: (1) Based on an annotated medical HSI data set (9059 images from 46 pigs), we present a tissue atlas featuring spectral fingerprints of 20 different porcine organs and tissue types. (2) Using the principle of mixed model analysis, we show that the greatest source of variability related to HSI images is the organ under observation. (3) We show that HSI-based fully-automatic tissue differentiation of 20 organ classes with deep neural networks is possible with high accuracy (> 95%). We conclude from our study that automatic tissue discrimination based on HSI data is feasible and could thus aid in intraoperative decisionmaking and pave the way for context-aware computer-assisted surgery systems and autonomous robotics.https://doi.org/10.1038/s41598-022-15040-w |
| spellingShingle | Alexander Studier-Fischer Silvia Seidlitz Jan Sellner Berkin Özdemir Manuel Wiesenfarth Leonardo Ayala Jan Odenthal Samuel Knödler Karl Friedrich Kowalewski Caelan Max Haney Isabella Camplisson Maximilian Dietrich Karsten Schmidt Gabriel Alexander Salg Hannes Götz Kenngott Tim Julian Adler Nicholas Schreck Annette Kopp-Schneider Klaus Maier-Hein Lena Maier-Hein Beat Peter Müller-Stich Felix Nickel Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model Scientific Reports |
| title | Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| title_full | Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| title_fullStr | Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| title_full_unstemmed | Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| title_short | Spectral organ fingerprints for machine learning-based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| title_sort | spectral organ fingerprints for machine learning based intraoperative tissue classification with hyperspectral imaging in a porcine model |
| url | https://doi.org/10.1038/s41598-022-15040-w |
| work_keys_str_mv | AT alexanderstudierfischer spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT silviaseidlitz spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT jansellner spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT berkinozdemir spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT manuelwiesenfarth spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT leonardoayala spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT janodenthal spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT samuelknodler spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT karlfriedrichkowalewski spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT caelanmaxhaney spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT isabellacamplisson spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT maximiliandietrich spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT karstenschmidt spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT gabrielalexandersalg spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT hannesgotzkenngott spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT timjulianadler spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT nicholasschreck spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT annettekoppschneider spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT klausmaierhein spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT lenamaierhein spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT beatpetermullerstich spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel AT felixnickel spectralorganfingerprintsformachinelearningbasedintraoperativetissueclassificationwithhyperspectralimaginginaporcinemodel |