GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization
Abstract Objects in a real scene often occlude each other and inferring a complete appearance from the visible part is an important and challenging task. In this paper, the authors propose a self‐supervised generative adversarial network GIGAN (GAN for generating the invisible), which can generate t...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-07-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12824 |
| _version_ | 1797786083755294720 |
|---|---|
| author | Fengnan Quan Bo Lang |
| author_facet | Fengnan Quan Bo Lang |
| author_sort | Fengnan Quan |
| collection | DOAJ |
| description | Abstract Objects in a real scene often occlude each other and inferring a complete appearance from the visible part is an important and challenging task. In this paper, the authors propose a self‐supervised generative adversarial network GIGAN (GAN for generating the invisible), which can generate the complete appearance of objects without labelled invisible part information. The authors build two cycle transformation networks CycleIncomplete (CycleI) and CycleComplete (CycleC) that share parameters to improve the accuracy of mask completion. This design does not require well‐matched training images and can make better use of the limited labelled samples. In addition, the authors propose a conditional normalization module and combine it with the inferred complete mask output. The combination not only enhances the content recovery ability and obtains more realistic outputs, but also improves the efficiency of the generation process. Experimental results show that compared with existing self‐supervised learning models, our method achieves l1 error, mean intersection‐over‐union (mIOU), and Fréchet inception distance (FID) improvements on the COCOA and KINS datasets. |
| first_indexed | 2024-03-13T01:03:57Z |
| format | Article |
| id | doaj.art-7517d9b7f30b4a4197f6cd683f9230a0 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-13T01:03:57Z |
| publishDate | 2023-07-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-7517d9b7f30b4a4197f6cd683f9230a02023-07-06T09:05:42ZengWileyIET Image Processing1751-96591751-96672023-07-011792736274710.1049/ipr2.12824GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalizationFengnan Quan0Bo Lang1State Key Laboratory of Software Development Environment Beihang University Beijing P. R. ChinaState Key Laboratory of Software Development Environment Beihang University Beijing P. R. ChinaAbstract Objects in a real scene often occlude each other and inferring a complete appearance from the visible part is an important and challenging task. In this paper, the authors propose a self‐supervised generative adversarial network GIGAN (GAN for generating the invisible), which can generate the complete appearance of objects without labelled invisible part information. The authors build two cycle transformation networks CycleIncomplete (CycleI) and CycleComplete (CycleC) that share parameters to improve the accuracy of mask completion. This design does not require well‐matched training images and can make better use of the limited labelled samples. In addition, the authors propose a conditional normalization module and combine it with the inferred complete mask output. The combination not only enhances the content recovery ability and obtains more realistic outputs, but also improves the efficiency of the generation process. Experimental results show that compared with existing self‐supervised learning models, our method achieves l1 error, mean intersection‐over‐union (mIOU), and Fréchet inception distance (FID) improvements on the COCOA and KINS datasets.https://doi.org/10.1049/ipr2.12824image processingimage segmentation |
| spellingShingle | Fengnan Quan Bo Lang GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization IET Image Processing image processing image segmentation |
| title | GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization |
| title_full | GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization |
| title_fullStr | GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization |
| title_full_unstemmed | GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization |
| title_short | GIGAN: Self‐supervised GAN for generating the invisible using cycle transformation and conditional normalization |
| title_sort | gigan self supervised gan for generating the invisible using cycle transformation and conditional normalization |
| topic | image processing image segmentation |
| url | https://doi.org/10.1049/ipr2.12824 |
| work_keys_str_mv | AT fengnanquan giganselfsupervisedganforgeneratingtheinvisibleusingcycletransformationandconditionalnormalization AT bolang giganselfsupervisedganforgeneratingtheinvisibleusingcycletransformationandconditionalnormalization |