In-Sensor Visual Perception and Inference
Conventional machine vision systems have separate perception, memory, and processing architectures, which may exacerbate the increasing need for ultrahigh image processing rates and ultralow power consumption. In contrast, in-sensor visual computing performs signal processing at the pixel level usin...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Association for the Advancement of Science (AAAS)
2023-01-01
|
| Series: | Intelligent Computing |
| Online Access: | https://spj.science.org/doi/10.34133/icomputing.0043 |
| _version_ | 1797672946601295872 |
|---|---|
| author | Yanan Liu Rui Fan Jianglong Guo Hepeng Ni Muhammad Usman Maqboo Bhutta |
| author_facet | Yanan Liu Rui Fan Jianglong Guo Hepeng Ni Muhammad Usman Maqboo Bhutta |
| author_sort | Yanan Liu |
| collection | DOAJ |
| description | Conventional machine vision systems have separate perception, memory, and processing architectures, which may exacerbate the increasing need for ultrahigh image processing rates and ultralow power consumption. In contrast, in-sensor visual computing performs signal processing at the pixel level using the collected analog signals directly, without sending data to other processors. Therefore, the in-sensor computing paradigm may hold the key to realizing extremely efficient and low power visual signal processing by integrating sensing, storage, and computation onto focal planes using either novel circuit designs or new materials. The focal-plane sensor-processor (FPSP), which is a typical in-sensor visual computing device, is a vision chip that has been developed for nearly 2 decades in domains such as image processing, computer vision, robotics, and neural networks. In contrast to conventional computer vision systems, the FPSP gives vision systems in-sensor image processing capabilities, thus decreasing system complexity, reducing power consumption, and enhancing information processing efficiency and security. Although many studies on in-sensor computing using the FPSP have been conducted since its invention, no thorough and systematic summary of these studies exists. This review explains the use of image processing algorithms, neural networks, and applications of in-sensor computing in the fields of machine vision and robotics. The objective is to assist future developers, researchers, and users of unconventional visual sensors in understanding in-sensor computing and associated applications. |
| first_indexed | 2024-03-11T21:37:31Z |
| format | Article |
| id | doaj.art-24002f118a514b8c8e6e92fa9bfabd4d |
| institution | Directory Open Access Journal |
| issn | 2771-5892 |
| language | English |
| last_indexed | 2024-03-11T21:37:31Z |
| publishDate | 2023-01-01 |
| publisher | American Association for the Advancement of Science (AAAS) |
| record_format | Article |
| series | Intelligent Computing |
| spelling | doaj.art-24002f118a514b8c8e6e92fa9bfabd4d2023-09-26T17:08:11ZengAmerican Association for the Advancement of Science (AAAS)Intelligent Computing2771-58922023-01-01210.34133/icomputing.0043In-Sensor Visual Perception and InferenceYanan Liu0Rui Fan1Jianglong Guo2Hepeng Ni3Muhammad Usman Maqboo Bhutta4School of Microelectronics, Shanghai University, Shanghai,China.The College of Electronics and Information Engineering, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University, Shanghai, China.School of Science, Harbin Institute of Technology, Shenzhen, China.School of Mechanical and Electrical Engineering, Shandong Jianzhu University, Jinan, China.Robotics Lab, The Chinese University of Hong Kong, Hong Kong, China.Conventional machine vision systems have separate perception, memory, and processing architectures, which may exacerbate the increasing need for ultrahigh image processing rates and ultralow power consumption. In contrast, in-sensor visual computing performs signal processing at the pixel level using the collected analog signals directly, without sending data to other processors. Therefore, the in-sensor computing paradigm may hold the key to realizing extremely efficient and low power visual signal processing by integrating sensing, storage, and computation onto focal planes using either novel circuit designs or new materials. The focal-plane sensor-processor (FPSP), which is a typical in-sensor visual computing device, is a vision chip that has been developed for nearly 2 decades in domains such as image processing, computer vision, robotics, and neural networks. In contrast to conventional computer vision systems, the FPSP gives vision systems in-sensor image processing capabilities, thus decreasing system complexity, reducing power consumption, and enhancing information processing efficiency and security. Although many studies on in-sensor computing using the FPSP have been conducted since its invention, no thorough and systematic summary of these studies exists. This review explains the use of image processing algorithms, neural networks, and applications of in-sensor computing in the fields of machine vision and robotics. The objective is to assist future developers, researchers, and users of unconventional visual sensors in understanding in-sensor computing and associated applications.https://spj.science.org/doi/10.34133/icomputing.0043 |
| spellingShingle | Yanan Liu Rui Fan Jianglong Guo Hepeng Ni Muhammad Usman Maqboo Bhutta In-Sensor Visual Perception and Inference Intelligent Computing |
| title | In-Sensor Visual Perception and Inference |
| title_full | In-Sensor Visual Perception and Inference |
| title_fullStr | In-Sensor Visual Perception and Inference |
| title_full_unstemmed | In-Sensor Visual Perception and Inference |
| title_short | In-Sensor Visual Perception and Inference |
| title_sort | in sensor visual perception and inference |
| url | https://spj.science.org/doi/10.34133/icomputing.0043 |
| work_keys_str_mv | AT yananliu insensorvisualperceptionandinference AT ruifan insensorvisualperceptionandinference AT jianglongguo insensorvisualperceptionandinference AT hepengni insensorvisualperceptionandinference AT muhammadusmanmaqboobhutta insensorvisualperceptionandinference |