Multi-task self-supervised visual learning
We investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling-in order to train a single visual representation. First, we provide an apples-to-apples comparison of four different self-supervised tasks using the very dee...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Published: |
IEEE Explore
2017
|
| _version_ | 1826305683930218496 |
|---|---|
| author | Doersch, C Zisserman, A |
| author_facet | Doersch, C Zisserman, A |
| author_sort | Doersch, C |
| collection | OXFORD |
| description | We investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling-in order to train a single visual representation. First, we provide an apples-to-apples comparison of four different self-supervised tasks using the very deep ResNet-101 architecture. We then combine tasks to jointly train a network. We also explore lasso regularization to encourage the network to factorize the information in its representation, and methods for “harmonizing” network inputs in order to learn a more unified representation. We evaluate all methods on ImageNet classification, PASCAL VOC detection, and NYU depth prediction. Our results show that deeper networks work better, and that combining tasks-even via a näýve multihead architecture-always improves performance. Our best joint network nearly matches the PASCAL performance of a model pre-trained on ImageNet classification, and matches the ImageNet network on NYU depth prediction. |
| first_indexed | 2024-03-07T06:36:33Z |
| format | Conference item |
| id | oxford-uuid:f7dca138-b143-4c6a-a96a-c79faba3388c |
| institution | University of Oxford |
| last_indexed | 2024-03-07T06:36:33Z |
| publishDate | 2017 |
| publisher | IEEE Explore |
| record_format | dspace |
| spelling | oxford-uuid:f7dca138-b143-4c6a-a96a-c79faba3388c2022-03-27T12:45:42ZMulti-task self-supervised visual learningConference itemhttp://purl.org/coar/resource_type/c_5794uuid:f7dca138-b143-4c6a-a96a-c79faba3388cSymplectic Elements at OxfordIEEE Explore2017Doersch, CZisserman, AWe investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling-in order to train a single visual representation. First, we provide an apples-to-apples comparison of four different self-supervised tasks using the very deep ResNet-101 architecture. We then combine tasks to jointly train a network. We also explore lasso regularization to encourage the network to factorize the information in its representation, and methods for “harmonizing” network inputs in order to learn a more unified representation. We evaluate all methods on ImageNet classification, PASCAL VOC detection, and NYU depth prediction. Our results show that deeper networks work better, and that combining tasks-even via a näýve multihead architecture-always improves performance. Our best joint network nearly matches the PASCAL performance of a model pre-trained on ImageNet classification, and matches the ImageNet network on NYU depth prediction. |
| spellingShingle | Doersch, C Zisserman, A Multi-task self-supervised visual learning |
| title | Multi-task self-supervised visual learning |
| title_full | Multi-task self-supervised visual learning |
| title_fullStr | Multi-task self-supervised visual learning |
| title_full_unstemmed | Multi-task self-supervised visual learning |
| title_short | Multi-task self-supervised visual learning |
| title_sort | multi task self supervised visual learning |
| work_keys_str_mv | AT doerschc multitaskselfsupervisedvisuallearning AT zissermana multitaskselfsupervisedvisuallearning |