UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features
Virtual Try-on is the ability to realistically superimpose clothing onto a target person. Due to its importance to the multi-billion dollar e-commerce industry, the problem has received significant attention in recent years. To date, most virtual try-on methods have been supervised approaches, namel...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/19/5647 |
| _version_ | 1827704963583705088 |
|---|---|
| author | Hideki Tsunashima Kosuke Arase Antony Lam Hirokatsu Kataoka |
| author_facet | Hideki Tsunashima Kosuke Arase Antony Lam Hirokatsu Kataoka |
| author_sort | Hideki Tsunashima |
| collection | DOAJ |
| description | Virtual Try-on is the ability to realistically superimpose clothing onto a target person. Due to its importance to the multi-billion dollar e-commerce industry, the problem has received significant attention in recent years. To date, most virtual try-on methods have been supervised approaches, namely using annotated data, such as clothes parsing semantic segmentation masks and paired images. These approaches incur a very high cost in annotation. Even existing weakly-supervised virtual try-on methods still use annotated data or pre-trained networks as auxiliary information and the costs of the annotation are still significantly high. Plus, the strategy using pre-trained networks is not appropriate in the practical scenarios due to latency. In this paper we propose Unsupervised VIRtual Try-on using disentangled representation (UVIRT). After UVIRT extracts a clothes and a person feature from a person image and a clothes image respectively, it exchanges a clothes and a person feature. Finally, UVIRT achieve virtual try-on. This is all achieved in an unsupervised manner so UVIRT has the advantage that it does not require any annotated data, pre-trained networks nor even category labels. In the experiments, we qualitatively and quantitatively compare between supervised methods and our UVIRT method on the MPV dataset (which has paired images) and on a Consumer-to-Consumer (C2C) marketplace dataset (which has unpaired images). As a result, UVIRT outperform the supervised method on the C2C marketplace dataset, and achieve comparable results on the MPV dataset, which has paired images in comparison with the conventional supervised method. |
| first_indexed | 2024-03-10T15:52:38Z |
| format | Article |
| id | doaj.art-39dec57f4e78495a87bc8f6356bccdbb |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T15:52:38Z |
| publishDate | 2020-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-39dec57f4e78495a87bc8f6356bccdbb2023-11-20T15:54:43ZengMDPI AGSensors1424-82202020-10-012019564710.3390/s20195647UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person FeaturesHideki Tsunashima0Kosuke Arase1Antony Lam2Hirokatsu Kataoka3Computer Vision Research Team, Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, JapanMercari, Inc., Tokyo 106-6188, JapanMercari, Inc., Tokyo 106-6188, JapanComputer Vision Research Team, Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, JapanVirtual Try-on is the ability to realistically superimpose clothing onto a target person. Due to its importance to the multi-billion dollar e-commerce industry, the problem has received significant attention in recent years. To date, most virtual try-on methods have been supervised approaches, namely using annotated data, such as clothes parsing semantic segmentation masks and paired images. These approaches incur a very high cost in annotation. Even existing weakly-supervised virtual try-on methods still use annotated data or pre-trained networks as auxiliary information and the costs of the annotation are still significantly high. Plus, the strategy using pre-trained networks is not appropriate in the practical scenarios due to latency. In this paper we propose Unsupervised VIRtual Try-on using disentangled representation (UVIRT). After UVIRT extracts a clothes and a person feature from a person image and a clothes image respectively, it exchanges a clothes and a person feature. Finally, UVIRT achieve virtual try-on. This is all achieved in an unsupervised manner so UVIRT has the advantage that it does not require any annotated data, pre-trained networks nor even category labels. In the experiments, we qualitatively and quantitatively compare between supervised methods and our UVIRT method on the MPV dataset (which has paired images) and on a Consumer-to-Consumer (C2C) marketplace dataset (which has unpaired images). As a result, UVIRT outperform the supervised method on the C2C marketplace dataset, and achieve comparable results on the MPV dataset, which has paired images in comparison with the conventional supervised method.https://www.mdpi.com/1424-8220/20/19/5647virtual try-onimage-to-image translationunsupervised learningGANdisentanglement |
| spellingShingle | Hideki Tsunashima Kosuke Arase Antony Lam Hirokatsu Kataoka UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features Sensors virtual try-on image-to-image translation unsupervised learning GAN disentanglement |
| title | UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features |
| title_full | UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features |
| title_fullStr | UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features |
| title_full_unstemmed | UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features |
| title_short | UVIRT—Unsupervised Virtual Try-on Using Disentangled Clothing and Person Features |
| title_sort | uvirt unsupervised virtual try on using disentangled clothing and person features |
| topic | virtual try-on image-to-image translation unsupervised learning GAN disentanglement |
| url | https://www.mdpi.com/1424-8220/20/19/5647 |
| work_keys_str_mv | AT hidekitsunashima uvirtunsupervisedvirtualtryonusingdisentangledclothingandpersonfeatures AT kosukearase uvirtunsupervisedvirtualtryonusingdisentangledclothingandpersonfeatures AT antonylam uvirtunsupervisedvirtualtryonusingdisentangledclothingandpersonfeatures AT hirokatsukataoka uvirtunsupervisedvirtualtryonusingdisentangledclothingandpersonfeatures |