Multimodal AutoML via Representation Evolution
With the increasing amounts of available data, learning simultaneously from different types of inputs is becoming necessary to obtain robust and well-performing models. With the advent of representation learning in recent years, lower-dimensional vector-based representations have become available fo...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Machine Learning and Knowledge Extraction |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-4990/5/1/1 |
| _version_ | 1797858349960658944 |
|---|---|
| author | Blaž Škrlj Matej Bevec Nada Lavrač |
| author_facet | Blaž Škrlj Matej Bevec Nada Lavrač |
| author_sort | Blaž Škrlj |
| collection | DOAJ |
| description | With the increasing amounts of available data, learning simultaneously from different types of inputs is becoming necessary to obtain robust and well-performing models. With the advent of representation learning in recent years, lower-dimensional vector-based representations have become available for both images and texts, while automating simultaneous learning from multiple modalities remains a challenging problem. This paper presents an AutoML (automated machine learning) approach to automated machine learning model configuration identification for data composed of two modalities: texts and images. The approach is based on the idea of representation evolution, the process of automatically amplifying heterogeneous representations across several modalities, optimized jointly with a collection of fast, well-regularized linear models. The proposed approach is benchmarked against 11 unimodal and multimodal (texts and images) approaches on four real-life benchmark datasets from different domains. It achieves competitive performance with minimal human effort and low computing requirements, enabling learning from multiple modalities in automated manner for a wider community of researchers. |
| first_indexed | 2024-04-09T21:13:00Z |
| format | Article |
| id | doaj.art-6b8718ffe6cc481b8bc9653da310d48e |
| institution | Directory Open Access Journal |
| issn | 2504-4990 |
| language | English |
| last_indexed | 2024-04-09T21:13:00Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machine Learning and Knowledge Extraction |
| spelling | doaj.art-6b8718ffe6cc481b8bc9653da310d48e2023-03-28T14:05:55ZengMDPI AGMachine Learning and Knowledge Extraction2504-49902022-12-015111310.3390/make5010001Multimodal AutoML via Representation EvolutionBlaž Škrlj0Matej Bevec1Nada Lavrač2Department of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000 Ljubljana, SloveniaDepartment of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000 Ljubljana, SloveniaDepartment of Knowledge Technologies, Jožef Stefan Institute, Jamova 39, 1000 Ljubljana, SloveniaWith the increasing amounts of available data, learning simultaneously from different types of inputs is becoming necessary to obtain robust and well-performing models. With the advent of representation learning in recent years, lower-dimensional vector-based representations have become available for both images and texts, while automating simultaneous learning from multiple modalities remains a challenging problem. This paper presents an AutoML (automated machine learning) approach to automated machine learning model configuration identification for data composed of two modalities: texts and images. The approach is based on the idea of representation evolution, the process of automatically amplifying heterogeneous representations across several modalities, optimized jointly with a collection of fast, well-regularized linear models. The proposed approach is benchmarked against 11 unimodal and multimodal (texts and images) approaches on four real-life benchmark datasets from different domains. It achieves competitive performance with minimal human effort and low computing requirements, enabling learning from multiple modalities in automated manner for a wider community of researchers.https://www.mdpi.com/2504-4990/5/1/1AutoMLrepresentation learningevolutionmultimodal learning |
| spellingShingle | Blaž Škrlj Matej Bevec Nada Lavrač Multimodal AutoML via Representation Evolution Machine Learning and Knowledge Extraction AutoML representation learning evolution multimodal learning |
| title | Multimodal AutoML via Representation Evolution |
| title_full | Multimodal AutoML via Representation Evolution |
| title_fullStr | Multimodal AutoML via Representation Evolution |
| title_full_unstemmed | Multimodal AutoML via Representation Evolution |
| title_short | Multimodal AutoML via Representation Evolution |
| title_sort | multimodal automl via representation evolution |
| topic | AutoML representation learning evolution multimodal learning |
| url | https://www.mdpi.com/2504-4990/5/1/1 |
| work_keys_str_mv | AT blazskrlj multimodalautomlviarepresentationevolution AT matejbevec multimodalautomlviarepresentationevolution AT nadalavrac multimodalautomlviarepresentationevolution |