Novel Multimodal, Multiscale Imaging System with Augmented Reality
A novel multimodal, multiscale imaging system with augmented reality capability were developed and characterized. The system offers 3D color reflectance imaging, 3D fluorescence imaging, and augmented reality in real time. Multiscale fluorescence imaging was enabled by developing and integrating an...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-03-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/11/3/441 |
| _version_ | 1797414408253603840 |
|---|---|
| author | Christopher Mela Francis Papay Yang Liu |
| author_facet | Christopher Mela Francis Papay Yang Liu |
| author_sort | Christopher Mela |
| collection | DOAJ |
| description | A novel multimodal, multiscale imaging system with augmented reality capability were developed and characterized. The system offers 3D color reflectance imaging, 3D fluorescence imaging, and augmented reality in real time. Multiscale fluorescence imaging was enabled by developing and integrating an in vivo fiber-optic microscope. Real-time ultrasound-fluorescence multimodal imaging used optically tracked fiducial markers for registration. Tomographical data are also incorporated using optically tracked fiducial markers for registration. Furthermore, we characterized system performance and registration accuracy in a benchtop setting. The multiscale fluorescence imaging facilitated assessing the functional status of tissues, extending the minimal resolution of fluorescence imaging to ~17.5 µm. The system achieved a mean of Target Registration error of less than 2 mm for registering fluorescence images to ultrasound images and MRI-based 3D model, which is within clinically acceptable range. The low latency and high frame rate of the prototype system has shown the promise of applying the reported techniques in clinically relevant settings in the future. |
| first_indexed | 2024-03-09T05:32:38Z |
| format | Article |
| id | doaj.art-c6c89a064b3048e4a3d8b9ef665ebe03 |
| institution | Directory Open Access Journal |
| issn | 2075-4418 |
| language | English |
| last_indexed | 2024-03-09T05:32:38Z |
| publishDate | 2021-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj.art-c6c89a064b3048e4a3d8b9ef665ebe032023-12-03T12:31:30ZengMDPI AGDiagnostics2075-44182021-03-0111344110.3390/diagnostics11030441Novel Multimodal, Multiscale Imaging System with Augmented RealityChristopher Mela0Francis Papay1Yang Liu2Iowa Technology Institute, The University of Iowa, 103 South Capitol Street, Iowa City, IA 52242, USADermatology & Plastic Surgery Institute, Cleveland Clinic, Cleveland, OH 44195, USAIowa Technology Institute, The University of Iowa, 103 South Capitol Street, Iowa City, IA 52242, USAA novel multimodal, multiscale imaging system with augmented reality capability were developed and characterized. The system offers 3D color reflectance imaging, 3D fluorescence imaging, and augmented reality in real time. Multiscale fluorescence imaging was enabled by developing and integrating an in vivo fiber-optic microscope. Real-time ultrasound-fluorescence multimodal imaging used optically tracked fiducial markers for registration. Tomographical data are also incorporated using optically tracked fiducial markers for registration. Furthermore, we characterized system performance and registration accuracy in a benchtop setting. The multiscale fluorescence imaging facilitated assessing the functional status of tissues, extending the minimal resolution of fluorescence imaging to ~17.5 µm. The system achieved a mean of Target Registration error of less than 2 mm for registering fluorescence images to ultrasound images and MRI-based 3D model, which is within clinically acceptable range. The low latency and high frame rate of the prototype system has shown the promise of applying the reported techniques in clinically relevant settings in the future.https://www.mdpi.com/2075-4418/11/3/441multimodal imagingmultiscale imagingcomputer visionaugmented realityfluorescence imaging |
| spellingShingle | Christopher Mela Francis Papay Yang Liu Novel Multimodal, Multiscale Imaging System with Augmented Reality Diagnostics multimodal imaging multiscale imaging computer vision augmented reality fluorescence imaging |
| title | Novel Multimodal, Multiscale Imaging System with Augmented Reality |
| title_full | Novel Multimodal, Multiscale Imaging System with Augmented Reality |
| title_fullStr | Novel Multimodal, Multiscale Imaging System with Augmented Reality |
| title_full_unstemmed | Novel Multimodal, Multiscale Imaging System with Augmented Reality |
| title_short | Novel Multimodal, Multiscale Imaging System with Augmented Reality |
| title_sort | novel multimodal multiscale imaging system with augmented reality |
| topic | multimodal imaging multiscale imaging computer vision augmented reality fluorescence imaging |
| url | https://www.mdpi.com/2075-4418/11/3/441 |
| work_keys_str_mv | AT christophermela novelmultimodalmultiscaleimagingsystemwithaugmentedreality AT francispapay novelmultimodalmultiscaleimagingsystemwithaugmentedreality AT yangliu novelmultimodalmultiscaleimagingsystemwithaugmentedreality |