Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer
Most CycleGAN domain transfer architectures require a large amount of data belonging to domains on which the domain transfer task is to be applied. Nevertheless, in many real-world applications one of the domains is reduced, i.e., scarce. This means that it has much less training data available in c...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/7/3411 |
| _version_ | 1797440509251158016 |
|---|---|
| author | Lidija Krstanović Branislav Popović Marko Janev Branko Brkljač |
| author_facet | Lidija Krstanović Branislav Popović Marko Janev Branko Brkljač |
| author_sort | Lidija Krstanović |
| collection | DOAJ |
| description | Most CycleGAN domain transfer architectures require a large amount of data belonging to domains on which the domain transfer task is to be applied. Nevertheless, in many real-world applications one of the domains is reduced, i.e., scarce. This means that it has much less training data available in comparison to the other domain, which is fully observable. In order to tackle the problem of using CycleGAN framework in such unfavorable application scenarios, we propose and invoke a novel Bootstrapped SSL CycleGAN architecture (BTS-SSL), where the mentioned problem is overcome using two strategies. Firstly, by using a relatively small percentage of available labelled training data from the reduced or scarce domain and a Semi-Supervised Learning (SSL) approach, we prevent overfitting of the discriminator belonging to the reduced domain, which would otherwise occur during initial training iterations due to the small amount of available training data in the scarce domain. Secondly, after initial learning guided by the described SSL strategy, additional bootstrapping (BTS) of the reduced data domain is performed by inserting artifically generated training examples into the training poll of the data discriminator belonging to the scarce domain. Bootstrapped samples are generated by the already trained neural network that performs transferring from the fully observable to the scarce domain. The described procedure is periodically repeated during the training process several times and results in significantly improved performance of the final model in comparison to the original unsupervised CycleGAN approach. The same also holds in comparison to the solutions that are exclusively based either on the described SSL, or on the bootstrapping strategy, i.e., when these are applied separately. Moreover, in the considered scarce scenarios it also shows competitive results in comparison to the fully supervised solution based on the pix2pix method. In that sense, it is directly applicable to many domain transfer tasks that are relying on the CycleGAN architecture. |
| first_indexed | 2024-03-09T12:09:12Z |
| format | Article |
| id | doaj.art-f708f82e64ff428bb3b9a4f809ec658a |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T12:09:12Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-f708f82e64ff428bb3b9a4f809ec658a2023-11-30T22:55:21ZengMDPI AGApplied Sciences2076-34172022-03-01127341110.3390/app12073411Bootstrapped SSL CycleGAN for Asymmetric Domain TransferLidija Krstanović0Branislav Popović1Marko Janev2Branko Brkljač3Faculty of Technical Sciences, University of Novi Sad, Trg D. Obradovića 6, 21000 Novi Sad, SerbiaFaculty of Technical Sciences, University of Novi Sad, Trg D. Obradovića 6, 21000 Novi Sad, SerbiaInstitute of Mathematics, Serbian Academy of Sciences and Arts, Kneza Mihaila 36, 11000 Belgrade, SerbiaFaculty of Technical Sciences, University of Novi Sad, Trg D. Obradovića 6, 21000 Novi Sad, SerbiaMost CycleGAN domain transfer architectures require a large amount of data belonging to domains on which the domain transfer task is to be applied. Nevertheless, in many real-world applications one of the domains is reduced, i.e., scarce. This means that it has much less training data available in comparison to the other domain, which is fully observable. In order to tackle the problem of using CycleGAN framework in such unfavorable application scenarios, we propose and invoke a novel Bootstrapped SSL CycleGAN architecture (BTS-SSL), where the mentioned problem is overcome using two strategies. Firstly, by using a relatively small percentage of available labelled training data from the reduced or scarce domain and a Semi-Supervised Learning (SSL) approach, we prevent overfitting of the discriminator belonging to the reduced domain, which would otherwise occur during initial training iterations due to the small amount of available training data in the scarce domain. Secondly, after initial learning guided by the described SSL strategy, additional bootstrapping (BTS) of the reduced data domain is performed by inserting artifically generated training examples into the training poll of the data discriminator belonging to the scarce domain. Bootstrapped samples are generated by the already trained neural network that performs transferring from the fully observable to the scarce domain. The described procedure is periodically repeated during the training process several times and results in significantly improved performance of the final model in comparison to the original unsupervised CycleGAN approach. The same also holds in comparison to the solutions that are exclusively based either on the described SSL, or on the bootstrapping strategy, i.e., when these are applied separately. Moreover, in the considered scarce scenarios it also shows competitive results in comparison to the fully supervised solution based on the pix2pix method. In that sense, it is directly applicable to many domain transfer tasks that are relying on the CycleGAN architecture.https://www.mdpi.com/2076-3417/12/7/3411CycleGAN architecturesemi-supervised learningbootstrappingimbalanced data |
| spellingShingle | Lidija Krstanović Branislav Popović Marko Janev Branko Brkljač Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer Applied Sciences CycleGAN architecture semi-supervised learning bootstrapping imbalanced data |
| title | Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer |
| title_full | Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer |
| title_fullStr | Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer |
| title_full_unstemmed | Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer |
| title_short | Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer |
| title_sort | bootstrapped ssl cyclegan for asymmetric domain transfer |
| topic | CycleGAN architecture semi-supervised learning bootstrapping imbalanced data |
| url | https://www.mdpi.com/2076-3417/12/7/3411 |
| work_keys_str_mv | AT lidijakrstanovic bootstrappedsslcycleganforasymmetricdomaintransfer AT branislavpopovic bootstrappedsslcycleganforasymmetricdomaintransfer AT markojanev bootstrappedsslcycleganforasymmetricdomaintransfer AT brankobrkljac bootstrappedsslcycleganforasymmetricdomaintransfer |