An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA
Due to the absence of communication and coordination with external spacecraft, non-cooperative spacecraft present challenges for the servicing spacecraft in acquiring information about their pose and location. The accurate segmentation of non-cooperative spacecraft components in images is a crucial...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/5/894 |
| _version_ | 1797263912944533504 |
|---|---|
| author | Zibo Guo Kai Liu Wei Liu Xiaoyao Sun Chongyang Ding Shangrong Li |
| author_facet | Zibo Guo Kai Liu Wei Liu Xiaoyao Sun Chongyang Ding Shangrong Li |
| author_sort | Zibo Guo |
| collection | DOAJ |
| description | Due to the absence of communication and coordination with external spacecraft, non-cooperative spacecraft present challenges for the servicing spacecraft in acquiring information about their pose and location. The accurate segmentation of non-cooperative spacecraft components in images is a crucial step in autonomously sensing the pose of non-cooperative spacecraft. This paper presents a novel overlay accelerator of DeepLab Convolutional Neural Networks (CNNs) for spacecraft image segmentation on a FPGA. First, several software–hardware co-design aspects are investigated: (1) A CNNs-domain COD instruction set (Control, Operation, Data Transfer) is presented based on a Load–Store architecture to enable the implementation of accelerator overlays. (2) An RTL-based prototype accelerator is developed for the COD instruction set. The accelerator incorporates dedicated units for instruction decoding and dispatch, scheduling, memory management, and operation execution. (3) A compiler is designed that leverages tiling and operation fusion techniques to optimize the execution of CNNs, generating binary instructions for the optimized operations. Our accelerator is implemented on a Xilinx Virtex-7 XC7VX690T FPGA at 200 MHz. Experiments demonstrate that with INT16 quantization our accelerator achieves an accuracy (mIoU) of 77.84%, experiencing only a 0.2% degradation compared to that of the original fully precision model, in accelerating the segmentation model of DeepLabv3+ ResNet18 on the spacecraft component images (SCIs) dataset. The accelerator boasts a performance of 184.19 GOPS/s and a computational efficiency (Runtime Throughput/Theoretical Roof Throughput) of 88.72%. Compared to previous work, our accelerator improves performance by 1.5× and computational efficiency by 43.93%, all while consuming similar hardware resources. Additionally, in terms of instruction encoding, our instructions reduce the size by 1.5× to 49× when compiling the same model compared to previous work. |
| first_indexed | 2024-04-25T00:20:33Z |
| format | Article |
| id | doaj.art-e20dd2e078f14ac6937997119c71020a |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-04-25T00:20:33Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-e20dd2e078f14ac6937997119c71020a2024-03-12T16:54:21ZengMDPI AGRemote Sensing2072-42922024-03-0116589410.3390/rs16050894An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGAZibo Guo0Kai Liu1Wei Liu2Xiaoyao Sun3Chongyang Ding4Shangrong Li5School of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSmart Earth Key Laboratory, Beijing 100094, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaSchool of Computer Science and Technology, Xidian University, Xi’an 710071, ChinaDue to the absence of communication and coordination with external spacecraft, non-cooperative spacecraft present challenges for the servicing spacecraft in acquiring information about their pose and location. The accurate segmentation of non-cooperative spacecraft components in images is a crucial step in autonomously sensing the pose of non-cooperative spacecraft. This paper presents a novel overlay accelerator of DeepLab Convolutional Neural Networks (CNNs) for spacecraft image segmentation on a FPGA. First, several software–hardware co-design aspects are investigated: (1) A CNNs-domain COD instruction set (Control, Operation, Data Transfer) is presented based on a Load–Store architecture to enable the implementation of accelerator overlays. (2) An RTL-based prototype accelerator is developed for the COD instruction set. The accelerator incorporates dedicated units for instruction decoding and dispatch, scheduling, memory management, and operation execution. (3) A compiler is designed that leverages tiling and operation fusion techniques to optimize the execution of CNNs, generating binary instructions for the optimized operations. Our accelerator is implemented on a Xilinx Virtex-7 XC7VX690T FPGA at 200 MHz. Experiments demonstrate that with INT16 quantization our accelerator achieves an accuracy (mIoU) of 77.84%, experiencing only a 0.2% degradation compared to that of the original fully precision model, in accelerating the segmentation model of DeepLabv3+ ResNet18 on the spacecraft component images (SCIs) dataset. The accelerator boasts a performance of 184.19 GOPS/s and a computational efficiency (Runtime Throughput/Theoretical Roof Throughput) of 88.72%. Compared to previous work, our accelerator improves performance by 1.5× and computational efficiency by 43.93%, all while consuming similar hardware resources. Additionally, in terms of instruction encoding, our instructions reduce the size by 1.5× to 49× when compiling the same model compared to previous work.https://www.mdpi.com/2072-4292/16/5/894image semantic segmentationinstruction set architecture (ISA)field programmable gate array (FPGA)spacecraft component images |
| spellingShingle | Zibo Guo Kai Liu Wei Liu Xiaoyao Sun Chongyang Ding Shangrong Li An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA Remote Sensing image semantic segmentation instruction set architecture (ISA) field programmable gate array (FPGA) spacecraft component images |
| title | An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA |
| title_full | An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA |
| title_fullStr | An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA |
| title_full_unstemmed | An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA |
| title_short | An Overlay Accelerator of DeepLab CNN for Spacecraft Image Segmentation on FPGA |
| title_sort | overlay accelerator of deeplab cnn for spacecraft image segmentation on fpga |
| topic | image semantic segmentation instruction set architecture (ISA) field programmable gate array (FPGA) spacecraft component images |
| url | https://www.mdpi.com/2072-4292/16/5/894 |
| work_keys_str_mv | AT ziboguo anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT kailiu anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT weiliu anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT xiaoyaosun anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT chongyangding anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT shangrongli anoverlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT ziboguo overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT kailiu overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT weiliu overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT xiaoyaosun overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT chongyangding overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga AT shangrongli overlayacceleratorofdeeplabcnnforspacecraftimagesegmentationonfpga |