Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks
Abstract Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems, evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evoluti...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer US
2021
|
| Online Access: | https://hdl.handle.net/1721.1/131981 |
| _version_ | 1811080702874091520 |
|---|---|
| author | Chaturvedi, Iti Su, Chit L Welsch, Roy E |
| author_facet | Chaturvedi, Iti Su, Chit L Welsch, Roy E |
| author_sort | Chaturvedi, Iti |
| collection | MIT |
| description | Abstract
Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems, evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evolution is achieved via selective imitation where two individuals with the same type of skill are encouraged to crossover. Due to the relatedness of the tasks, the resulting offspring may have a skill for a different task. In this way, we can simultaneously evolve a population where different individuals excel in different tasks. In this paper, we consider a type of evolution called Genetic Programming (GP) where the population of genes have a tree-like structure and can be of different lengths and hence can naturally represent multiple tasks. We apply the model to multi-task neuroevolution that aims to determine the optimal hyper-parameters of a neural network such as number of nodes, learning rate, and number of training epochs using evolution. Here each gene is encoded with the hyper parameters for a single neural network. Previously, optimization was done by enabling or disabling individual connections between neurons during evolution. This method is extremely slow and does not generalize well to new neural architectures such as Seq2Seq. To overcome this limitation, we follow a modular approach where each sub-tree in a GP can be a sub-neural architecture that is preserved during crossover across multiple tasks. Lastly, in order to leverage on the inter-task covariance for faster evolutionary search, we project the features from both tasks to common space using fuzzy membership functions. The proposed model is used to determine the optimal topology of a feed-forward neural network for classification of emotions in physiological heart signals and also a Seq2seq chatbot that can converse with kindergarten children. We can outperform baselines by over 10% in accuracy. |
| first_indexed | 2024-09-23T11:35:25Z |
| format | Article |
| id | mit-1721.1/131981 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:35:25Z |
| publishDate | 2021 |
| publisher | Springer US |
| record_format | dspace |
| spelling | mit-1721.1/1319812022-01-19T04:06:02Z Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks Chaturvedi, Iti Su, Chit L Welsch, Roy E Abstract Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems, evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evolution is achieved via selective imitation where two individuals with the same type of skill are encouraged to crossover. Due to the relatedness of the tasks, the resulting offspring may have a skill for a different task. In this way, we can simultaneously evolve a population where different individuals excel in different tasks. In this paper, we consider a type of evolution called Genetic Programming (GP) where the population of genes have a tree-like structure and can be of different lengths and hence can naturally represent multiple tasks. We apply the model to multi-task neuroevolution that aims to determine the optimal hyper-parameters of a neural network such as number of nodes, learning rate, and number of training epochs using evolution. Here each gene is encoded with the hyper parameters for a single neural network. Previously, optimization was done by enabling or disabling individual connections between neurons during evolution. This method is extremely slow and does not generalize well to new neural architectures such as Seq2Seq. To overcome this limitation, we follow a modular approach where each sub-tree in a GP can be a sub-neural architecture that is preserved during crossover across multiple tasks. Lastly, in order to leverage on the inter-task covariance for faster evolutionary search, we project the features from both tasks to common space using fuzzy membership functions. The proposed model is used to determine the optimal topology of a feed-forward neural network for classification of emotions in physiological heart signals and also a Seq2seq chatbot that can converse with kindergarten children. We can outperform baselines by over 10% in accuracy. 2021-09-20T17:41:14Z 2021-09-20T17:41:14Z 2021-01-05 2021-01-21T04:28:19Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/131981 en https://doi.org/10.1007/s12559-020-09807-4 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature application/pdf Springer US Springer US |
| spellingShingle | Chaturvedi, Iti Su, Chit L Welsch, Roy E Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title | Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title_full | Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title_fullStr | Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title_full_unstemmed | Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title_short | Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks |
| title_sort | fuzzy aggregated topology evolution for cognitive multi tasks |
| url | https://hdl.handle.net/1721.1/131981 |
| work_keys_str_mv | AT chaturvediiti fuzzyaggregatedtopologyevolutionforcognitivemultitasks AT suchitl fuzzyaggregatedtopologyevolutionforcognitivemultitasks AT welschroye fuzzyaggregatedtopologyevolutionforcognitivemultitasks |