Content-Attribute Disentanglement for Generalized Zero-Shot Learning
Humans can recognize or infer unseen classes of objects using descriptions explaining the characteristics (semantic information) of the classes. However, conventional deep learning models trained in a supervised manner cannot classify classes that were unseen during training. Hence, many studies hav...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9784824/ |
| _version_ | 1811243383484579840 |
|---|---|
| author | Yoojin An Sangyeon Kim Yuxuan Liang Roger Zimmermann Dongho Kim Jihie Kim |
| author_facet | Yoojin An Sangyeon Kim Yuxuan Liang Roger Zimmermann Dongho Kim Jihie Kim |
| author_sort | Yoojin An |
| collection | DOAJ |
| description | Humans can recognize or infer unseen classes of objects using descriptions explaining the characteristics (semantic information) of the classes. However, conventional deep learning models trained in a supervised manner cannot classify classes that were unseen during training. Hence, many studies have been conducted into generalized zero-shot learning (GZSL), which aims to produce system which can recognize both seen and unseen classes, by transferring learned knowledge from seen to unseen classes. Since seen and unseen classes share a common semantic space, extracting appropriate semantic information from images is essential for GZSL. In addition to semantic-related information (attributes), images also contain semantic-unrelated information (contents), which can degrade the classification performance of the model. Therefore, we propose a content-attribute disentanglement architecture which separates the content and attribute information of images. The proposed method is comprised of three major components: 1) a feature generation module for synthesizing unseen visual features; 2) a content-attribute disentanglement module for discriminating content and attribute codes from images; and 3) an attribute comparator module for measuring the compatibility between the attribute codes and the class prototypes which act as the ground truth. With extensive experiments, we show that our method achieves state-of-the-art and competitive results on four benchmark datasets in GZSL. Our method also outperforms the existing zero-shot learning methods in all of the datasets. Moreover, our method has the best accuracy as well in a zero-shot retrieval task. Our code is available at <uri>https://github.com/anyoojin1996/CA-GZSL</uri>. |
| first_indexed | 2024-04-12T14:06:29Z |
| format | Article |
| id | doaj.art-7ec6f5c994b54a23920d0bb6a0748186 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T14:06:29Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-7ec6f5c994b54a23920d0bb6a07481862022-12-22T03:30:03ZengIEEEIEEE Access2169-35362022-01-0110583205833110.1109/ACCESS.2022.31788009784824Content-Attribute Disentanglement for Generalized Zero-Shot LearningYoojin An0https://orcid.org/0000-0001-6703-7789Sangyeon Kim1https://orcid.org/0000-0003-0717-0240Yuxuan Liang2Roger Zimmermann3https://orcid.org/0000-0002-7410-2590Dongho Kim4Jihie Kim5https://orcid.org/0000-0003-2358-4021Department of Artificial Intelligence, Dongguk University, Seoul, South KoreaNAVER WEBTOON AI, Seongnam, South KoreaSchool of Computing, National University of Singapore, SingaporeSchool of Computing, National University of Singapore, SingaporeDongguk Institute of Convergence Education, Dongguk University, Seoul, South KoreaDepartment of Artificial Intelligence, Dongguk University, Seoul, South KoreaHumans can recognize or infer unseen classes of objects using descriptions explaining the characteristics (semantic information) of the classes. However, conventional deep learning models trained in a supervised manner cannot classify classes that were unseen during training. Hence, many studies have been conducted into generalized zero-shot learning (GZSL), which aims to produce system which can recognize both seen and unseen classes, by transferring learned knowledge from seen to unseen classes. Since seen and unseen classes share a common semantic space, extracting appropriate semantic information from images is essential for GZSL. In addition to semantic-related information (attributes), images also contain semantic-unrelated information (contents), which can degrade the classification performance of the model. Therefore, we propose a content-attribute disentanglement architecture which separates the content and attribute information of images. The proposed method is comprised of three major components: 1) a feature generation module for synthesizing unseen visual features; 2) a content-attribute disentanglement module for discriminating content and attribute codes from images; and 3) an attribute comparator module for measuring the compatibility between the attribute codes and the class prototypes which act as the ground truth. With extensive experiments, we show that our method achieves state-of-the-art and competitive results on four benchmark datasets in GZSL. Our method also outperforms the existing zero-shot learning methods in all of the datasets. Moreover, our method has the best accuracy as well in a zero-shot retrieval task. Our code is available at <uri>https://github.com/anyoojin1996/CA-GZSL</uri>.https://ieeexplore.ieee.org/document/9784824/Computer visiondeep learningdisentangled representationgeneralized zero-shot learning |
| spellingShingle | Yoojin An Sangyeon Kim Yuxuan Liang Roger Zimmermann Dongho Kim Jihie Kim Content-Attribute Disentanglement for Generalized Zero-Shot Learning IEEE Access Computer vision deep learning disentangled representation generalized zero-shot learning |
| title | Content-Attribute Disentanglement for Generalized Zero-Shot Learning |
| title_full | Content-Attribute Disentanglement for Generalized Zero-Shot Learning |
| title_fullStr | Content-Attribute Disentanglement for Generalized Zero-Shot Learning |
| title_full_unstemmed | Content-Attribute Disentanglement for Generalized Zero-Shot Learning |
| title_short | Content-Attribute Disentanglement for Generalized Zero-Shot Learning |
| title_sort | content attribute disentanglement for generalized zero shot learning |
| topic | Computer vision deep learning disentangled representation generalized zero-shot learning |
| url | https://ieeexplore.ieee.org/document/9784824/ |
| work_keys_str_mv | AT yoojinan contentattributedisentanglementforgeneralizedzeroshotlearning AT sangyeonkim contentattributedisentanglementforgeneralizedzeroshotlearning AT yuxuanliang contentattributedisentanglementforgeneralizedzeroshotlearning AT rogerzimmermann contentattributedisentanglementforgeneralizedzeroshotlearning AT donghokim contentattributedisentanglementforgeneralizedzeroshotlearning AT jihiekim contentattributedisentanglementforgeneralizedzeroshotlearning |