Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network
The novel rotating synthetic aperture (RSA) is a new optical imaging system that uses the method of rotating the rectangular primary mirror for dynamic imaging. It has the advantage of being lightweight, with no need for splicing and real-time surface shape maintenance on orbit. The novel imaging me...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/3/688 |
| _version_ | 1797623309228048384 |
|---|---|
| author | Yu Sun Xiyang Zhi Shikai Jiang Jinnan Gong Tianjun Shi Nan Wang |
| author_facet | Yu Sun Xiyang Zhi Shikai Jiang Jinnan Gong Tianjun Shi Nan Wang |
| author_sort | Yu Sun |
| collection | DOAJ |
| description | The novel rotating synthetic aperture (RSA) is a new optical imaging system that uses the method of rotating the rectangular primary mirror for dynamic imaging. It has the advantage of being lightweight, with no need for splicing and real-time surface shape maintenance on orbit. The novel imaging method leads to complex image quality degradation characteristics. Therefore, it is vital to use the image quality improvement method to restore and improve the image quality to meet the application requirements. For the RSA system, a new system that has not been applied in orbit, it is difficult to construct suitable large datasets. Therefore, it is necessary to study and establish the dynamic imaging characteristic model of the RSA system, and on this basis provide data support for the corresponding image super resolution and restoration method through simulation. In this paper, we first analyze the imaging characteristics and mathematically model the rectangular rotary pupil of the RSA system. On this basis, combined with the analysis of the physical interpretation of the blur kernel, we find that the optimal blur kernel is not the point spread function (PSF) of the imaging system. Therefore, the simulation method of convolving the input image directly with the PSF is flawed. Furthermore, the weights of a convolutional neural network (CNN) are the same for each input. This means that the normal convolutional layer is not only difficult to accurately estimate the time-varying blur kernel, but also difficult to adapt to the change in the length–width ratio of the primary mirror. To that end, we propose a blur kernel estimation conditional convolutional neural network (CCNN) that is equivalent to multiple normal CNNs. We extend the CNN to a conditional model by taking an encoding as an additional input and using conditionally parameterized convolutions instead of normal convolutions. The CCNN can simulate the imaging characteristics of the rectangular pupil with different length–width ratios and different rotation angles in a controllable manner. The results of semi-physical experiments show that the proposed simulation method achieves a satisfactory performance, which can provide data and theoretical support for the image restoration and super-resolution method of the RSA system. |
| first_indexed | 2024-03-11T09:26:51Z |
| format | Article |
| id | doaj.art-272576a189b84fb6b75ccda356c16c0e |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T09:26:51Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-272576a189b84fb6b75ccda356c16c0e2023-11-16T17:52:47ZengMDPI AGRemote Sensing2072-42922023-01-0115368810.3390/rs15030688Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural NetworkYu Sun0Xiyang Zhi1Shikai Jiang2Jinnan Gong3Tianjun Shi4Nan Wang5Research Center for Space Optical Engineering, Harbin Institute of Technology, Harbin 150001, ChinaResearch Center for Space Optical Engineering, Harbin Institute of Technology, Harbin 150001, ChinaResearch Center for Space Optical Engineering, Harbin Institute of Technology, Harbin 150001, ChinaResearch Center for Space Optical Engineering, Harbin Institute of Technology, Harbin 150001, ChinaResearch Center for Space Optical Engineering, Harbin Institute of Technology, Harbin 150001, ChinaForeign Studies College of Northeastern University, Shenyang 110001, ChinaThe novel rotating synthetic aperture (RSA) is a new optical imaging system that uses the method of rotating the rectangular primary mirror for dynamic imaging. It has the advantage of being lightweight, with no need for splicing and real-time surface shape maintenance on orbit. The novel imaging method leads to complex image quality degradation characteristics. Therefore, it is vital to use the image quality improvement method to restore and improve the image quality to meet the application requirements. For the RSA system, a new system that has not been applied in orbit, it is difficult to construct suitable large datasets. Therefore, it is necessary to study and establish the dynamic imaging characteristic model of the RSA system, and on this basis provide data support for the corresponding image super resolution and restoration method through simulation. In this paper, we first analyze the imaging characteristics and mathematically model the rectangular rotary pupil of the RSA system. On this basis, combined with the analysis of the physical interpretation of the blur kernel, we find that the optimal blur kernel is not the point spread function (PSF) of the imaging system. Therefore, the simulation method of convolving the input image directly with the PSF is flawed. Furthermore, the weights of a convolutional neural network (CNN) are the same for each input. This means that the normal convolutional layer is not only difficult to accurately estimate the time-varying blur kernel, but also difficult to adapt to the change in the length–width ratio of the primary mirror. To that end, we propose a blur kernel estimation conditional convolutional neural network (CCNN) that is equivalent to multiple normal CNNs. We extend the CNN to a conditional model by taking an encoding as an additional input and using conditionally parameterized convolutions instead of normal convolutions. The CCNN can simulate the imaging characteristics of the rectangular pupil with different length–width ratios and different rotation angles in a controllable manner. The results of semi-physical experiments show that the proposed simulation method achieves a satisfactory performance, which can provide data and theoretical support for the image restoration and super-resolution method of the RSA system.https://www.mdpi.com/2072-4292/15/3/688optical remote sensingimage simulationrotating synthetic aperturerectangular pupilconditional convolutional neural network |
| spellingShingle | Yu Sun Xiyang Zhi Shikai Jiang Jinnan Gong Tianjun Shi Nan Wang Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network Remote Sensing optical remote sensing image simulation rotating synthetic aperture rectangular pupil conditional convolutional neural network |
| title | Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network |
| title_full | Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network |
| title_fullStr | Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network |
| title_full_unstemmed | Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network |
| title_short | Imaging Simulation Method for Novel Rotating Synthetic Aperture System Based on Conditional Convolutional Neural Network |
| title_sort | imaging simulation method for novel rotating synthetic aperture system based on conditional convolutional neural network |
| topic | optical remote sensing image simulation rotating synthetic aperture rectangular pupil conditional convolutional neural network |
| url | https://www.mdpi.com/2072-4292/15/3/688 |
| work_keys_str_mv | AT yusun imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork AT xiyangzhi imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork AT shikaijiang imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork AT jinnangong imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork AT tianjunshi imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork AT nanwang imagingsimulationmethodfornovelrotatingsyntheticaperturesystembasedonconditionalconvolutionalneuralnetwork |