Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction
An important challenge in machine learning is performing with accuracy when few training samples are available from the target distribution. If a large number of training samples from a related distribution are available, transfer learning can be used to improve the performance. This paper investiga...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/25/11/1554 |
| _version_ | 1797459396648763392 |
|---|---|
| author | Nathan Cornille Katrien Laenen Jingyuan Sun Marie-Francine Moens |
| author_facet | Nathan Cornille Katrien Laenen Jingyuan Sun Marie-Francine Moens |
| author_sort | Nathan Cornille |
| collection | DOAJ |
| description | An important challenge in machine learning is performing with accuracy when few training samples are available from the target distribution. If a large number of training samples from a related distribution are available, transfer learning can be used to improve the performance. This paper investigates how to do transfer learning more effectively if the source and target distributions are related through a Sparse Mechanism Shift for the application of next-frame prediction. We create Sparse Mechanism Shift-TempoRal Intervened Sequences (SMS-TRIS), a benchmark to evaluate transfer learning for next-frame prediction derived from the TRIS datasets. We then propose to exploit the Sparse Mechanism Shift property of the distribution shift by disentangling the model parameters with regard to the true causal mechanisms underlying the data. We use the Causal Identifiability from TempoRal Intervened Sequences (CITRIS) model to achieve this disentanglement via causal representation learning. We show that encouraging disentanglement with the CITRIS extensions can improve performance, but their effectiveness varies depending on the dataset and backbone used. We find that it is effective only when encouraging disentanglement actually succeeds in increasing disentanglement. We also show that an alternative method designed for domain adaptation does not help, indicating the challenging nature of the SMS-TRIS benchmark. |
| first_indexed | 2024-03-09T16:50:45Z |
| format | Article |
| id | doaj.art-47949f389e03478ab6d2fc0f3cc74a48 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T16:50:45Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-47949f389e03478ab6d2fc0f3cc74a482023-11-24T14:41:07ZengMDPI AGEntropy1099-43002023-11-012511155410.3390/e25111554Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video PredictionNathan Cornille0Katrien Laenen1Jingyuan Sun2Marie-Francine Moens3Language Intelligence and Information Retrieval (LIIR) Lab, Department of Computer Science KU Leuven, 3001 Leuven, BelgiumLanguage Intelligence and Information Retrieval (LIIR) Lab, Department of Computer Science KU Leuven, 3001 Leuven, BelgiumLanguage Intelligence and Information Retrieval (LIIR) Lab, Department of Computer Science KU Leuven, 3001 Leuven, BelgiumLanguage Intelligence and Information Retrieval (LIIR) Lab, Department of Computer Science KU Leuven, 3001 Leuven, BelgiumAn important challenge in machine learning is performing with accuracy when few training samples are available from the target distribution. If a large number of training samples from a related distribution are available, transfer learning can be used to improve the performance. This paper investigates how to do transfer learning more effectively if the source and target distributions are related through a Sparse Mechanism Shift for the application of next-frame prediction. We create Sparse Mechanism Shift-TempoRal Intervened Sequences (SMS-TRIS), a benchmark to evaluate transfer learning for next-frame prediction derived from the TRIS datasets. We then propose to exploit the Sparse Mechanism Shift property of the distribution shift by disentangling the model parameters with regard to the true causal mechanisms underlying the data. We use the Causal Identifiability from TempoRal Intervened Sequences (CITRIS) model to achieve this disentanglement via causal representation learning. We show that encouraging disentanglement with the CITRIS extensions can improve performance, but their effectiveness varies depending on the dataset and backbone used. We find that it is effective only when encouraging disentanglement actually succeeds in increasing disentanglement. We also show that an alternative method designed for domain adaptation does not help, indicating the challenging nature of the SMS-TRIS benchmark.https://www.mdpi.com/1099-4300/25/11/1554causal representation learningvideo predictiontransfer learningfew-shot learning |
| spellingShingle | Nathan Cornille Katrien Laenen Jingyuan Sun Marie-Francine Moens Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction Entropy causal representation learning video prediction transfer learning few-shot learning |
| title | Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction |
| title_full | Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction |
| title_fullStr | Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction |
| title_full_unstemmed | Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction |
| title_short | Causal Factor Disentanglement for Few-Shot Domain Adaptation in Video Prediction |
| title_sort | causal factor disentanglement for few shot domain adaptation in video prediction |
| topic | causal representation learning video prediction transfer learning few-shot learning |
| url | https://www.mdpi.com/1099-4300/25/11/1554 |
| work_keys_str_mv | AT nathancornille causalfactordisentanglementforfewshotdomainadaptationinvideoprediction AT katrienlaenen causalfactordisentanglementforfewshotdomainadaptationinvideoprediction AT jingyuansun causalfactordisentanglementforfewshotdomainadaptationinvideoprediction AT mariefrancinemoens causalfactordisentanglementforfewshotdomainadaptationinvideoprediction |