Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture
Lensless imaging represents a significant advancement in imaging technology, offering unique benefits over traditional optical systems due to its compact form factor, ideal for applications within the Internet of Things (IoT) ecosystem. Despite its potential, the intensive computational requirements...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Photonics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2304-6732/10/11/1274 |
| _version_ | 1797457983002640384 |
|---|---|
| author | Muyuan Liu Xiuqin Su Xiaopeng Yao Wei Hao Wenhua Zhu |
| author_facet | Muyuan Liu Xiuqin Su Xiaopeng Yao Wei Hao Wenhua Zhu |
| author_sort | Muyuan Liu |
| collection | DOAJ |
| description | Lensless imaging represents a significant advancement in imaging technology, offering unique benefits over traditional optical systems due to its compact form factor, ideal for applications within the Internet of Things (IoT) ecosystem. Despite its potential, the intensive computational requirements of current lensless imaging reconstruction algorithms pose a challenge, often exceeding the resource constraints typical of IoT devices. To meet this challenge, a novel approach is introduced, merging multi-level image restoration with the pix2pix generative adversarial network architecture within the lensless imaging sphere. Building on the foundation provided by U-Net, a Multi-level Attention-based Lensless Image Restoration Network (MARN) is introduced to further augment the generator’s capabilities. In this methodology, images reconstructed through Tikhonov regularization are perceived as degraded images, forming the foundation for further refinement via the Pix2pix network. This process is enhanced by incorporating an attention-focused mechanism in the encoder--decoder structure and by implementing stage-wise supervised training within the deep convolutional network, contributing markedly to the improvement of the final image quality. Through detailed comparative evaluations, the superiority of the introduced method is affirmed, outperforming existing techniques and underscoring its suitability for addressing the computational challenges in lensless imaging within IoT environments. This method can produce excellent lensless image reconstructions when sufficient computational resources are available, and it consistently delivers optimal results across varying computational resource constraints. This algorithm enhances the applicability of lensless imaging in applications such as the Internet of Things, providing higher-quality image acquisition and processing capabilities for these domains. |
| first_indexed | 2024-03-09T16:31:35Z |
| format | Article |
| id | doaj.art-99d8657badd1402fb06bd03ed8f59209 |
| institution | Directory Open Access Journal |
| issn | 2304-6732 |
| language | English |
| last_indexed | 2024-03-09T16:31:35Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Photonics |
| spelling | doaj.art-99d8657badd1402fb06bd03ed8f592092023-11-24T15:01:41ZengMDPI AGPhotonics2304-67322023-11-011011127410.3390/photonics10111274Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix ArchitectureMuyuan Liu0Xiuqin Su1Xiaopeng Yao2Wei Hao3Wenhua Zhu4Key Laboratory of Space Precision Measurement Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, ChinaKey Laboratory of Space Precision Measurement Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, ChinaKey Laboratory of Space Precision Measurement Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, ChinaKey Laboratory of Space Precision Measurement Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, ChinaSchool of Electronic and Information Engineering, Jiujiang University, Jiujiang 332005, ChinaLensless imaging represents a significant advancement in imaging technology, offering unique benefits over traditional optical systems due to its compact form factor, ideal for applications within the Internet of Things (IoT) ecosystem. Despite its potential, the intensive computational requirements of current lensless imaging reconstruction algorithms pose a challenge, often exceeding the resource constraints typical of IoT devices. To meet this challenge, a novel approach is introduced, merging multi-level image restoration with the pix2pix generative adversarial network architecture within the lensless imaging sphere. Building on the foundation provided by U-Net, a Multi-level Attention-based Lensless Image Restoration Network (MARN) is introduced to further augment the generator’s capabilities. In this methodology, images reconstructed through Tikhonov regularization are perceived as degraded images, forming the foundation for further refinement via the Pix2pix network. This process is enhanced by incorporating an attention-focused mechanism in the encoder--decoder structure and by implementing stage-wise supervised training within the deep convolutional network, contributing markedly to the improvement of the final image quality. Through detailed comparative evaluations, the superiority of the introduced method is affirmed, outperforming existing techniques and underscoring its suitability for addressing the computational challenges in lensless imaging within IoT environments. This method can produce excellent lensless image reconstructions when sufficient computational resources are available, and it consistently delivers optimal results across varying computational resource constraints. This algorithm enhances the applicability of lensless imaging in applications such as the Internet of Things, providing higher-quality image acquisition and processing capabilities for these domains.https://www.mdpi.com/2304-6732/10/11/1274lensless imagingpix2piximage restorationmulti-stage deep neural network |
| spellingShingle | Muyuan Liu Xiuqin Su Xiaopeng Yao Wei Hao Wenhua Zhu Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture Photonics lensless imaging pix2pix image restoration multi-stage deep neural network |
| title | Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture |
| title_full | Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture |
| title_fullStr | Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture |
| title_full_unstemmed | Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture |
| title_short | Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture |
| title_sort | lensless image restoration based on multi stage deep neural networks and pix2pix architecture |
| topic | lensless imaging pix2pix image restoration multi-stage deep neural network |
| url | https://www.mdpi.com/2304-6732/10/11/1274 |
| work_keys_str_mv | AT muyuanliu lenslessimagerestorationbasedonmultistagedeepneuralnetworksandpix2pixarchitecture AT xiuqinsu lenslessimagerestorationbasedonmultistagedeepneuralnetworksandpix2pixarchitecture AT xiaopengyao lenslessimagerestorationbasedonmultistagedeepneuralnetworksandpix2pixarchitecture AT weihao lenslessimagerestorationbasedonmultistagedeepneuralnetworksandpix2pixarchitecture AT wenhuazhu lenslessimagerestorationbasedonmultistagedeepneuralnetworksandpix2pixarchitecture |