Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution
ABSTRACT: Spectral imaging extends the concept of traditional color cameras to capture images across multiple spectral channels and has broad application prospects. Conventional spectral cameras based on scanning methods suffer from the drawbacks of low acquisition speed and large volume. On-chip co...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-06-01
|
| Series: | Chip |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2709472323000084 |
| _version_ | 1827374891602542592 |
|---|---|
| author | Jiawei Yang Kaiyu Cui Yidong Huang Wei Zhang Xue Feng Fang Liu |
| author_facet | Jiawei Yang Kaiyu Cui Yidong Huang Wei Zhang Xue Feng Fang Liu |
| author_sort | Jiawei Yang |
| collection | DOAJ |
| description | ABSTRACT: Spectral imaging extends the concept of traditional color cameras to capture images across multiple spectral channels and has broad application prospects. Conventional spectral cameras based on scanning methods suffer from the drawbacks of low acquisition speed and large volume. On-chip computational spectral imaging based on metasurface filters provides a promising scheme for portable applications, but endures long computation time due to point-by-point iterative spectral reconstruction and mosaic effect in the reconstructed spectral images. In this study, on-chip rapid spectral imaging was demonstrated, which eliminated the mosaic effect in the spectral image by deep-learning-based spectral data cube reconstruction. The experimental results show that 4 orders of magnitude faster than the iterative spectral reconstruction were achieved, and the fidelity of the spectral reconstruction for the standard color plate was over 99% for a standard color board. In particular, video-rate spectral imaging was demonstrated for moving objects and outdoor driving scenes with good performance for recognizing metamerism, where the concolorous sky and white cars can be distinguished via their spectra, showing great potential for autonomous driving and other practical applications in the field of intelligent perception. |
| first_indexed | 2024-03-08T11:41:41Z |
| format | Article |
| id | doaj.art-b241d22eb4c04ec0aa2e3518d1687d41 |
| institution | Directory Open Access Journal |
| issn | 2709-4723 |
| language | English |
| last_indexed | 2024-03-08T11:41:41Z |
| publishDate | 2023-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Chip |
| spelling | doaj.art-b241d22eb4c04ec0aa2e3518d1687d412024-01-25T05:24:03ZengElsevierChip2709-47232023-06-0122100045Deep‐learning based on‐chip rapid spectral imaging with high spatial resolutionJiawei Yang0Kaiyu Cui1Yidong Huang2Wei Zhang3Xue Feng4Fang Liu5Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China; Corresponding authors.Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China; Bejing Academy of Quantum Information Science, Beijing 100084, China; Corresponding authors.Department of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China; Bejing Academy of Quantum Information Science, Beijing 100084, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing 100084, China; Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, ChinaABSTRACT: Spectral imaging extends the concept of traditional color cameras to capture images across multiple spectral channels and has broad application prospects. Conventional spectral cameras based on scanning methods suffer from the drawbacks of low acquisition speed and large volume. On-chip computational spectral imaging based on metasurface filters provides a promising scheme for portable applications, but endures long computation time due to point-by-point iterative spectral reconstruction and mosaic effect in the reconstructed spectral images. In this study, on-chip rapid spectral imaging was demonstrated, which eliminated the mosaic effect in the spectral image by deep-learning-based spectral data cube reconstruction. The experimental results show that 4 orders of magnitude faster than the iterative spectral reconstruction were achieved, and the fidelity of the spectral reconstruction for the standard color plate was over 99% for a standard color board. In particular, video-rate spectral imaging was demonstrated for moving objects and outdoor driving scenes with good performance for recognizing metamerism, where the concolorous sky and white cars can be distinguished via their spectra, showing great potential for autonomous driving and other practical applications in the field of intelligent perception.http://www.sciencedirect.com/science/article/pii/S2709472323000084Spectral imagingDeep learningMetasurface |
| spellingShingle | Jiawei Yang Kaiyu Cui Yidong Huang Wei Zhang Xue Feng Fang Liu Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution Chip Spectral imaging Deep learning Metasurface |
| title | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution |
| title_full | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution |
| title_fullStr | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution |
| title_full_unstemmed | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution |
| title_short | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution |
| title_sort | deep learning based on chip rapid spectral imaging with high spatial resolution |
| topic | Spectral imaging Deep learning Metasurface |
| url | http://www.sciencedirect.com/science/article/pii/S2709472323000084 |
| work_keys_str_mv | AT jiaweiyang deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution AT kaiyucui deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution AT yidonghuang deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution AT weizhang deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution AT xuefeng deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution AT fangliu deeplearningbasedonchiprapidspectralimagingwithhighspatialresolution |