A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging
Abstract Advanced machine intelligence is empowered not only by the ever‐increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and co...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-05-01
|
| Series: | Advanced Science |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/advs.202206699 |
| _version_ | 1797825254258638848 |
|---|---|
| author | Hao Dong Xubin Zheng Chen Cheng Libin Qian Yaoxuan Cui Weiwei Wu Qingjun Liu Xing Chen Yanli Lu Qing Yang Fenni Zhang Di Wang |
| author_facet | Hao Dong Xubin Zheng Chen Cheng Libin Qian Yaoxuan Cui Weiwei Wu Qingjun Liu Xing Chen Yanli Lu Qing Yang Fenni Zhang Di Wang |
| author_sort | Hao Dong |
| collection | DOAJ |
| description | Abstract Advanced machine intelligence is empowered not only by the ever‐increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and complex data processing. Herein, it is shown that a complementary metal‐oxide‐semiconductor (CMOS) imager can be transformed into a compact multimodal sensing platform through dual‐focus imaging. By combining lens‐based and lensless imaging, visual information, chemicals, temperature, and humidity can be detected with the same chip and output as a single image. As a proof of concept, the sensor is equipped on a micro‐vehicle, and multimodal environmental sensing and mapping is demonstrated. A multimodal endoscope is also developed, and simultaneous imaging and chemical profiling along a porcine digestive tract is achieved. The multimodal CMOS imager is compact, versatile, and extensible and can be widely applied in microrobots, in vivo medical apparatuses, and other microdevices. |
| first_indexed | 2024-03-13T10:51:19Z |
| format | Article |
| id | doaj.art-71834f9a8822411ba873e468e1203dc3 |
| institution | Directory Open Access Journal |
| issn | 2198-3844 |
| language | English |
| last_indexed | 2024-03-13T10:51:19Z |
| publishDate | 2023-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Science |
| spelling | doaj.art-71834f9a8822411ba873e468e1203dc32023-05-17T13:02:05ZengWileyAdvanced Science2198-38442023-05-011014n/an/a10.1002/advs.202206699A Multimodal Sensing CMOS Imager Based on Dual‐Focus ImagingHao Dong0Xubin Zheng1Chen Cheng2Libin Qian3Yaoxuan Cui4Weiwei Wu5Qingjun Liu6Xing Chen7Yanli Lu8Qing Yang9Fenni Zhang10Di Wang11Intelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaSchool of Advanced Materials and Nanotechnology Interdisciplinary Research Center of Smart Sensors Xidian University Shaanxi 710126 ChinaBiosensor National Special Laboratory Key Laboratory for Biomedical Engineering of Education Ministry College of Biomedical Engineering and Instrument Science Zhejiang University Hangzhou 310027 ChinaBiosensor National Special Laboratory Key Laboratory for Biomedical Engineering of Education Ministry College of Biomedical Engineering and Instrument Science Zhejiang University Hangzhou 310027 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaBiosensor National Special Laboratory Key Laboratory for Biomedical Engineering of Education Ministry College of Biomedical Engineering and Instrument Science Zhejiang University Hangzhou 310027 ChinaIntelligent Perception Research Institute Zhejiang Lab Hangzhou 311100 ChinaAbstract Advanced machine intelligence is empowered not only by the ever‐increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and complex data processing. Herein, it is shown that a complementary metal‐oxide‐semiconductor (CMOS) imager can be transformed into a compact multimodal sensing platform through dual‐focus imaging. By combining lens‐based and lensless imaging, visual information, chemicals, temperature, and humidity can be detected with the same chip and output as a single image. As a proof of concept, the sensor is equipped on a micro‐vehicle, and multimodal environmental sensing and mapping is demonstrated. A multimodal endoscope is also developed, and simultaneous imaging and chemical profiling along a porcine digestive tract is achieved. The multimodal CMOS imager is compact, versatile, and extensible and can be widely applied in microrobots, in vivo medical apparatuses, and other microdevices.https://doi.org/10.1002/advs.202206699CMOS imagerscolorimetric arrayslab on chiplensless imagingmultimodal sensors |
| spellingShingle | Hao Dong Xubin Zheng Chen Cheng Libin Qian Yaoxuan Cui Weiwei Wu Qingjun Liu Xing Chen Yanli Lu Qing Yang Fenni Zhang Di Wang A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging Advanced Science CMOS imagers colorimetric arrays lab on chip lensless imaging multimodal sensors |
| title | A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging |
| title_full | A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging |
| title_fullStr | A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging |
| title_full_unstemmed | A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging |
| title_short | A Multimodal Sensing CMOS Imager Based on Dual‐Focus Imaging |
| title_sort | multimodal sensing cmos imager based on dual focus imaging |
| topic | CMOS imagers colorimetric arrays lab on chip lensless imaging multimodal sensors |
| url | https://doi.org/10.1002/advs.202206699 |
| work_keys_str_mv | AT haodong amultimodalsensingcmosimagerbasedondualfocusimaging AT xubinzheng amultimodalsensingcmosimagerbasedondualfocusimaging AT chencheng amultimodalsensingcmosimagerbasedondualfocusimaging AT libinqian amultimodalsensingcmosimagerbasedondualfocusimaging AT yaoxuancui amultimodalsensingcmosimagerbasedondualfocusimaging AT weiweiwu amultimodalsensingcmosimagerbasedondualfocusimaging AT qingjunliu amultimodalsensingcmosimagerbasedondualfocusimaging AT xingchen amultimodalsensingcmosimagerbasedondualfocusimaging AT yanlilu amultimodalsensingcmosimagerbasedondualfocusimaging AT qingyang amultimodalsensingcmosimagerbasedondualfocusimaging AT fennizhang amultimodalsensingcmosimagerbasedondualfocusimaging AT diwang amultimodalsensingcmosimagerbasedondualfocusimaging AT haodong multimodalsensingcmosimagerbasedondualfocusimaging AT xubinzheng multimodalsensingcmosimagerbasedondualfocusimaging AT chencheng multimodalsensingcmosimagerbasedondualfocusimaging AT libinqian multimodalsensingcmosimagerbasedondualfocusimaging AT yaoxuancui multimodalsensingcmosimagerbasedondualfocusimaging AT weiweiwu multimodalsensingcmosimagerbasedondualfocusimaging AT qingjunliu multimodalsensingcmosimagerbasedondualfocusimaging AT xingchen multimodalsensingcmosimagerbasedondualfocusimaging AT yanlilu multimodalsensingcmosimagerbasedondualfocusimaging AT qingyang multimodalsensingcmosimagerbasedondualfocusimaging AT fennizhang multimodalsensingcmosimagerbasedondualfocusimaging AT diwang multimodalsensingcmosimagerbasedondualfocusimaging |