Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application
In recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a ch...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/13/3130 |
| _version_ | 1797408240807444480 |
|---|---|
| author | Tianwei Yan Ning Zhang Jie Li Wenchao Liu He Chen |
| author_facet | Tianwei Yan Ning Zhang Jie Li Wenchao Liu He Chen |
| author_sort | Tianwei Yan |
| collection | DOAJ |
| description | In recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a challenging problem, especially for the spaceborne scenario where resources and power consumption are limited. To tackle this problem, this paper proposes an automatic CNN deployment solution on resource-limited field-programmable gate arrays (FPGAs) for spaceborne remote sensing applications. Firstly, a series of hardware-oriented optimization methods are proposed to reduce the complexity of the CNNs. Secondly, a hardware accelerator is designed. In this accelerator, a reconfigurable processing engine array with efficient convolutional computation architecture is used to accelerate CNN-based algorithms. Thirdly, to bridge the optimized CNNs and hardware accelerator, a compilation toolchain is introduced into the deployment solution. Through the automatic conversion from CNN models to hardware instructions, various networks can be deployed on hardware in real-time. Finally, we deployed an improved VGG16 network and an improved YOLOv2 network on Xilinx AC701 to evaluate the effectiveness of the proposed deployment solution. The experiments show that with only 3.407 W power consumption and 94 DSP consumption, our solution achieves 23.06 giga operations per second (GOPS) throughput in the improved VGG16 and 22.17 GOPS throughput in the improved YOLOv2. Compared to the related works, the DSP efficiency of our solution is improved by 1.3–2.7×. |
| first_indexed | 2024-03-09T03:55:34Z |
| format | Article |
| id | doaj.art-6842200f90ef44658612ed90232a3c4d |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T03:55:34Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-6842200f90ef44658612ed90232a3c4d2023-12-03T14:20:41ZengMDPI AGRemote Sensing2072-42922022-06-011413313010.3390/rs14133130Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing ApplicationTianwei Yan0Ning Zhang1Jie Li2Wenchao Liu3He Chen4Beijing Key Laboratory of Embedded Real-Time Information Processing Technology, Beijing Institute of Technology, Beijing 100081, ChinaBeijing Key Laboratory of Embedded Real-Time Information Processing Technology, Beijing Institute of Technology, Beijing 100081, ChinaInformation Processing Department, Shanghai Aerospace Electronic Technology Institute, Shanghai 201108, ChinaBeijing Key Laboratory of Embedded Real-Time Information Processing Technology, Beijing Institute of Technology, Beijing 100081, ChinaBeijing Key Laboratory of Embedded Real-Time Information Processing Technology, Beijing Institute of Technology, Beijing 100081, ChinaIn recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a challenging problem, especially for the spaceborne scenario where resources and power consumption are limited. To tackle this problem, this paper proposes an automatic CNN deployment solution on resource-limited field-programmable gate arrays (FPGAs) for spaceborne remote sensing applications. Firstly, a series of hardware-oriented optimization methods are proposed to reduce the complexity of the CNNs. Secondly, a hardware accelerator is designed. In this accelerator, a reconfigurable processing engine array with efficient convolutional computation architecture is used to accelerate CNN-based algorithms. Thirdly, to bridge the optimized CNNs and hardware accelerator, a compilation toolchain is introduced into the deployment solution. Through the automatic conversion from CNN models to hardware instructions, various networks can be deployed on hardware in real-time. Finally, we deployed an improved VGG16 network and an improved YOLOv2 network on Xilinx AC701 to evaluate the effectiveness of the proposed deployment solution. The experiments show that with only 3.407 W power consumption and 94 DSP consumption, our solution achieves 23.06 giga operations per second (GOPS) throughput in the improved VGG16 and 22.17 GOPS throughput in the improved YOLOv2. Compared to the related works, the DSP efficiency of our solution is improved by 1.3–2.7×.https://www.mdpi.com/2072-4292/14/13/3130remote sensingconvolutional neural networks (CNNs)optimizationfield-programmable gate array (FPGA)compilation toolchain |
| spellingShingle | Tianwei Yan Ning Zhang Jie Li Wenchao Liu He Chen Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application Remote Sensing remote sensing convolutional neural networks (CNNs) optimization field-programmable gate array (FPGA) compilation toolchain |
| title | Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application |
| title_full | Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application |
| title_fullStr | Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application |
| title_full_unstemmed | Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application |
| title_short | Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application |
| title_sort | automatic deployment of convolutional neural networks on fpga for spaceborne remote sensing application |
| topic | remote sensing convolutional neural networks (CNNs) optimization field-programmable gate array (FPGA) compilation toolchain |
| url | https://www.mdpi.com/2072-4292/14/13/3130 |
| work_keys_str_mv | AT tianweiyan automaticdeploymentofconvolutionalneuralnetworksonfpgaforspaceborneremotesensingapplication AT ningzhang automaticdeploymentofconvolutionalneuralnetworksonfpgaforspaceborneremotesensingapplication AT jieli automaticdeploymentofconvolutionalneuralnetworksonfpgaforspaceborneremotesensingapplication AT wenchaoliu automaticdeploymentofconvolutionalneuralnetworksonfpgaforspaceborneremotesensingapplication AT hechen automaticdeploymentofconvolutionalneuralnetworksonfpgaforspaceborneremotesensingapplication |